pipelines.results.orm module#

class pipelines.results.orm.OrmPipelineResultsStorage#

Bases: PipelineResultsStorage

Class to store pipeline results in the django ORM

build_pipeline_execution(pipeline: Pipeline, run_id: str, runner: PipelineRunner, reporter: PipelineReporter, input_data: Dict[str, Any], build_all=True)#

Creates a pipeline execution object along with all the pipeline results, task executions and task results.

If build_all is True, the storage class should create:

  • 1 pipeline execution object

  • A pipeline result object per element in the pipeline iterator

  • Per pipeline result, a tasks execution object per task in the pipeline

  • Per task execution object, a task result object per element in the task iterator

If build_all is False, only the pipeline execution should be created. This is used to capture error states when the pipeline is badly configured and the other objects may not be able to be created.

Parameters:

  • pipeline – The Pipeline object to create the results storage for

  • run_id – A UUID for referencing the run

  • runner – The PipelineRunner object that will be used to run the pipeline

  • reporter – The PipelineReporter object that will be used to report status changed

  • input_data – A json serializable dict containing the input parameters

  • build_all – If True, all results objects are create otherwise only the pipeline execution object will be created (to store config errors for example)

cleanup(before: datetime | None = None)#

Removes all results objects from the storage.

Parameters:

before – If set only objects created before the date will be removed. Otherwise all will be removed.

get_pipeline_digest() → Dict[str, PipelineDigestItem]#

Returns the PipelineDigest object providing stats for all registered pipeline classes.

get_pipeline_execution(run_id)#

Fetch a specific pipeline execution from the storage.

If the pipeline execution isn’t found, None will be returned.

Parameters:

run_id – The id of the pipeline run to fetch the execution for

get_pipeline_executions(pipeline_id: str | None = None)#

Gets all pipeline executions from the storage. If pipeline_id is supplied only executions for the given pipeline id will be returned

Parameters:

pipeline_id – The id of the registered pipeline class

get_pipeline_result(_id)#

Fetch a specific pipeline result from the storage.

If the pipeline result isn’t found, None will be returned.

Parameters:

_id – The id of the result to fetch from storage

get_pipeline_results(run_id: str | None = None) → Sequence[PipelineResult]#

Gets all pipeline results from the storage. If run_id is supplied only results for that particular run will be returned.

Parameters:

run_id – The id of the run to filter results by

get_task_execution(_id)#

Fetch a specific task execution from the storage.

If the pipeline result isn’t found, None will be returned.

Parameters:

_id – The id of the task execution to fetch from storage

get_task_executions(run_id: str | None = None, pipeline_result_id: str | None = None) → Sequence[TaskExecution]#

Gets all task executions from the storage.

Parameters:

  • run_id – The id of the run to filter results by

  • pipeline_result_id – The id of the parent pipeline result object to filter results by

get_task_result(_id)#

Fetch a specific task result from the storage.

If the pipeline result isn’t found, None will be returned.

Parameters:

_id – The id of the task result to fetch from storage

get_task_results(run_id: str | None = None, pipeline_result_id: str | None = None, task_execution_id: str | None = None) → Sequence[TaskResult]#

Gets all task results from the storage.

Parameters:

  • run_id – The id of the run to filter results by

  • pipeline_result_id – The id of the grandparent pipeline result object to filter results by

  • task_execution_id – The id of the parent task execution object to filter results by