Create a database table of past pipeline runs to populate/create data for schedules that have run.
model name: PipelineRun
fields:
pipeline (FK to Pipeline)
started_at (datetime with the date/time that the celery task started)
ended_at (datetime with the date/time that the celery task ended)
output (JSONField with the following schema: {"pipeline_id": pipeline_id, "steps": [{"step_id": step_id, "returncode": 0, "stdout": stdout_from_script, "stderr": stderr_from_script}, ...etc for each step in the pipeline]})
ronardcaktus
commented
2 years ago
Create a database table of past pipeline runs to populate/create data for schedules that have run.
PipelineRunpipeline(FK toPipeline)started_at(datetime with the date/time that the celery task started)ended_at(datetime with the date/time that the celery task ended)output(JSONField with the following schema:{"pipeline_id": pipeline_id, "steps": [{"step_id": step_id, "returncode": 0, "stdout": stdout_from_script, "stderr": stderr_from_script}, ...etc for each step in the pipeline]})