pipelines.runners.base module#
- class pipelines.runners.base.PipelineRunner#
Bases:
object
The base class to use when implementing a custom pipeline runner.
- classmethod get_flat_task_list(pipeline_result: PipelineResult) List[TaskExecution] #
Returns a list of all the tasks in an order they can be ran in order to conform to all runner ordering guarantees. :param pipeline_result: The pipeline result object to build the graph from
- classmethod get_task_graph(pipeline_result: PipelineResult) TopologicalSorter #
Returns a
TopologicalSorter
(seegraphlib.TopologicalSorter
) object representing the ordering of tasks in the pipeline.- Parameters:
pipeline_result – The pipeline result object to build the graph from
- run(pipeline_execution: PipelineExecution, reporter: PipelineReporter)#
Runs the pipeline. This should be reimplemented by each runner to schedule the pipeline as required.
Returns
True
if the pipeline was scheduled successfully,False
otherwise. This has no guarantee on the completion of the pipeline, just that it has been successfully scheduled.- Parameters:
pipeline_execution – The pipeline execution object representing the pipeline to run.
reporter – The reporter object to write messages to.
- start(pipeline_execution: PipelineExecution, reporter: PipelineReporter) bool #
Starts the pipeline running by the runner. This will pass the parameters off to the
run
method so the runner can handle the pipeline as it wishes.Returns
True
if the pipeline was scheduled successfully,False
otherwise. This has no guarantee on the completion of the pipeline, just that it has been successfully scheduled.- Parameters:
pipeline_execution – The pipeline execution object representing the pipeline to run.
reporter – The reporter object to write messages to.