Planning and Validation

Overview

This section covers how a workflow can be validated, planned, and simulated. Validation ensures your workflow is syntactically error-free, while planning computes a non-deterministic schedule of task execution, optimizing instrument or labware utilization.

Validation

Validation checks deterministic, a priori, and syntactic characteristics of a workflow’s structure to catch errors that would make a workflow impossible to plan. Additionally, it can optionally validate drivers and other configuration values by invoking validation for execution/driver validation.

Validation via python:

workflow = Workflow(...)

result = linq.validate_workflow(workflow) # optionally set validate_for_execution=True

Validation via CLI:

linq workflow validate <path of the file>

Optionally you can check if actions defined in the configuration are compatible with drivers:

linq workflow validate <path of the file> --validate-drivers

You can also check for time constraints etc.:

linq workflow validate <path of the file> --validate-infeasibility

What is validated?

Validation occurs in the LINQ Cloud environment and is constantly being updated to improve performance and breadth of checks. These are some of the workflow features that are validated:

Task dependencies are confirmed logical (non-circular, etc)
Time constraints are logically possible
Slots are logically specified and non-conflicting

With validation for execution/validating drivers:

Instrument configs are correct for a given driver
Driver versions exist
Tasks have the required configuration values
Transport is properly configured

Attention

Validation does not guarantee a workflow will run without issue. It only checks deterministic issues.

Planning

Planning is a non-deterministic process where the LINQ Scheduler takes a workflow definition and organizes it into a specific, optimized series of tasks. Resulting plans are the instruction set for the tasks to be executed on a workcell. These plans can be reviewed before execution. Doing so can help a user make better decisions about allocation, batching, or uncover subtle errors and misconfigurations.

Note

Planning successfully guarantees no deadlocks will occur!

The input to the planning process is a workflow definition, which includes tasks, constraints, and dependencies. The scheduling engine processes this input and generates an optimized plan. The output consists of the ordered sequence of tasks and the calculated time for each task, including transport times.

This section will cover how to invoke planning, the options available to control scheduling logic, and the artifacts produced by planning.

Creating a plan

The general lifecycle of initiating a workflow is as follows:

Validate
Create
Plan
Publish
Deploy
Start

Validate Workflow

linq workflow validate --path=quickstart.py --workflow=workflow

For details, see above.

Create Workflow

Creating a workflow stages the workflow in the LINQ API, making it available to the planner and other downstream functions.

linq workflow create --path=quickstart.py --workflow=workflow

This will return a workflow ID.

Note

linq workflow list can be used to view staged workflows. See linq workflow --help for more.

Start Plan

To start a plan, use:

linq workflow plan start --workflow-id=<workflow-id>

Check Plan Status

To check the current progress of your plan is available with:

linq workflow plan status --workflow-id=<workflow-id> --plan-id=<plan-id>

Plan statuses are PENDING, COMPLETED, or FAILED.

Retrieve Plan Result

Once the plan is complete, retrieve the task execution details with this command:

linq workflow plan result --workflow-id=<workflow-id> --plan-id=<plan-id>

Note

Use --watch to poll for updates. Use --silent to suppress terminal output.

Visualizations

To visualize a plans output in instrument-centric or labware-centric utilization gantt chart views, use --visualize with instrument or labware respectively.

Planner Options

The planner options allow you to fine-tune how the scheduler optimizes task sequencing and resource allocation. These settings give you greater control over precisely how a plan is generated.

options = Options(
    planner=PlannerOptions(
        target_objective = 7200,
        max_computation_time = 200,
        max_dead_time = 5,
        round_robin_load_balancing = True,
        preset = "PROD_LARGE",
        optimisation_target = "balance_total_plan_duration_and_labware_idle_time"
    )
)

class linq.workflow.Options(*, planner: PlannerOptions | None = None, executor: ExecutorOptions | None = None, is_explicit: bool = False)

Planning and execution options for a workflow.

planner: PlannerOptions | None: Planning-specific options, used to tweak what the planner optimises for.

executor: ExecutorOptions | None: Execution-specific options.

is_explicit: bool: Set this to true to prevent the API from adding any implicit logic, like opening doors, loading/unloading instruments, etc.

class linq.workflow.PlannerOptions(*, target_objective: float | None = None, max_computation_time: int | None = None, max_dead_time: int | None = None, round_robin_load_balancing: bool = False, preset: Literal['DRAFT', 'TEST', 'PROD_STANDARD', 'PROD_LARGE'] | None = None, optimisation_target: Literal['minimize_total_plan_duration', 'minimize_labware_idle_time', 'balance_total_plan_duration_and_labware_idle_time'] = 'minimize_total_plan_duration')

Planning-specific options, used to tweak how the planner behaves and what it optimises for.

target_objective: float | None: Target objective value.

max_computation_time: int | None: Max computation time allowed for the planner, in seconds. Starting point governed by option guaranteed_solution.

max_dead_time: int | None: Max dead time allowed between tasks with a transport between them

round_robin_load_balancing: bool: Where there are equidistant instruments (instruments that take the same time to transport to), maestro uses round robin to assign instruments to those tasks.

preset: Literal['DRAFT', 'TEST', 'PROD_STANDARD', 'PROD_LARGE'] | None: The plan CLI has a number of presets that can be used to quickly adjust the behaviour of the planner to suit different user needs. The available presets are: - DRAFT: To be used when it is not important if the plan is accurate or feasible. A plan generated with this preset does not guarantee that the workflow is actually possible to run in the real world - TEST: The planner will try to find a plan quickly, but it may not be optimal. - PROD_STANDARD: A preset with fairly standard settings. This is useful for making sure that the planner is using the recommended settings. - PROD_LARGE: A preset with settings that are more suitable for large workflows. This preset is useful when the workflow is large and the planner is taking a long time to find a plan.

More information about planning parameters

Target Objective: Defines the target objective for the planner’s optimisation. The default is minimize_total_plan_duration, meaning the planner will aim to minimise the total time it takes to execute the entire workflow.
Max Computation Time: Limits the amount of time the planner can spend generating the plan, in seconds.
Max Dead Time: Restricts the allowable idle time between tasks that involve transporting labware or items between instruments.
Round Robin Load Balancing: When there are multiple equidistant instruments (instruments that take the same time to transport to), this option allows the planner to assign tasks to those instruments in a round-robin manner. This helps distribute the workload more evenly.
Preset: The planner offers different presets that adjust its behavior based on your needs:
- "DRAFT": Quick plan generation without ensuring feasibility in real-world execution.
- "TEST": Faster but potentially non-optimal plans.
- "PROD_STANDARD": Standard settings for more reliable planning.
- "PROD_LARGE": Optimized for larger workflows that require more computation time.
Optimisation Target: Defines what the planner optimizes for:
- "minimize_total_plan_duration": Shortest total time.
- "minimize_labware_idle_time": Minimize labware wait times.
- "balance_total_plan_duration_and_labware_idle_time": Balance both.

Scheduler Versioning

Every workflow can be pinned to a specific version of the scheduler.

Note

A typical use case for selecting a scheduler version is to “freeze” a workflow and ensure execution is the same into the future. You can always use the latest scheduler on new workflows to take advantage of the latest features and optimizations.

To view the available scheduler versions, use linq scheduler-versions. This will list the different versions available. To select a specific version for your workflow, add the --schedule={VERSION} option at the end of above command.

Tip

Select the most recent scheduler version during workflow development in the SDK for a seamless development experience: scheduler_version=get_latest_scheduler_version()

Replanning

Replanning happens when the original plan no longer aligns with real-time execution. Replanning occurs in real time locally on a workcell. Typically, replans occur very quickly by leveraging the past plan solve for mathematical optimization. Replanning occurs due to:

Inaccurate Task Timings: If actual task durations differ from the defined task time estimates, the system automatically triggers replanning to generate a new schedule/adjust the schedule. This is the cause of the majority of observed replans.
Errors: When an error occurs and is resolved/recovered, whether manually or automatically, the lost time typically prompts the system to replan the workflow.
Conditionals: If a runtime conditional evaluates other than the expected outcome, a replan will be triggered.

Avoiding replanning

Replanning in and of itself is not necessarily a problem. However, to reduce replanning, ensure you set accurate Time Estimates.