search

genome / ptero-workflow

Client-facing API for the PTero system
1 stars 6 forks source link

Enable paging for execution data #236

Closed davidlmorton closed 8 years ago

davidlmorton commented 8 years ago

In order to be able to view the status of really large workflows we need to be able to fetch the data in bite-sized chunks.

In order for paging to be viable, you have to have an append-only data model. Lucky for us, we pretty much do.

1) Remove status_history from workflow_executions report 2) Remove child_workflows from workflow_executions report 3) Add creation timestamp to executions 4) Page response based on creation timestamp

5) Add workflow_execution_history report (which is Paged on timestamp. 6) Add child_workflows report (no need for paging probably)

It will be up to the client to re-assemble these paged responses and associate the ExecutionStatusHistory entries with the appropriate Execution.

davidlmorton commented 8 years ago

We probably want to create a paged_workflow_executions and paged_workflow_execution_status_history_entries reports rather than modifying the existing workflow_executions report behavior.