Project Builder: Workflow Steps cannot be updated

API issue?

Recorded on 6 Oct 23:00 BST Affects: FEM WFs with Transcription Tasks, possibly others

When the Project Builder attempts to send an update to the API, to change a Workflow's Steps, the data isn't fully digested (possibly by the API?). As a result, any Workflow that uses Steps (mostly WFs with Transcription Tasks) stop working.

For example, if I send...

PUT /api/workflows/3631

body: {
  workflows: {
    steps: [[ "S0", { "taskKeys": [ "T0", "T1" ] } ]]
  }
}

...the API will return...

workflow: {
  steps: [[ "S0", {} ]]
}

Original Report

@snblickhan reported on Slack that a Transcription WF on Corresponding with Quakers was crashing.

This WF 22387 was copied (via Project Builder's "Copy Workflow" function) from WF 22208.

@mcbouslog determined that the problem was due to 22208 having "incomplete steps". (i.e. you have step keys S1, but empty {} step values)

Replication

Open the project builder for any project with FEM Lab + Transcription Tasks enabled, e.g. https://master.pfe-preview.zooniverse.org/lab/1966/workflows/?env=staging
Create a new Workflow.
Add a Transcription Task.
Observe the WF that was updated in response, in your Network Tab.
- Expected: the steps data sent on the PUT (update) request should be reflected in the updated WF.
- Actual: the steps data in the updated WF is missing data.

Example:

On a newly-created Workflow, I create a Transcription Task, generating a PUT request.

The API, however, responds with steps that's missing data

Status

This is an urgent issue for any project FEM project with Transcription Tasks.

Impact:

No new Transcription WFs can be properly created for FEM. (WF will immediately crash on FEM Classifier)
Editing existing WFs with Transcription Tasks may cause problems to that WF.
Any Transcription WF created/edited while this bug exists will need to be manually fixed, after the bug is sorted?

Possible workaround:

for each Transcription WF affected, edit the WF resource's .steps to the correct value, directly via the database. (untested)

Looks like this is an underlying framework issue fixed in 5.2+ and we need to backport it to support this nested array form.

https://github.com/rails/rails/pull/32593/files that landed in https://github.com/rails/rails/blob/v5.2.8.1/actionpack/CHANGELOG.md#rails-521-august-07-2018

I'll look at backporting this fix for rails < 5.2

The issue is the payload structure of nested arrarys with json objects.

FWIW - i find the steps payload data structure somewhat strange, why is it an array of arrays instead of say a keyed json object? I'm most likely naive about what this data structure is trying to achieve but i think this would have worked without issue

# payload that doesn't work
[[ "S0", { "taskKeys": [ "T0", "T1" ] } ]]

# payload that should work where the step identifier links the task steps...
{ "S0": [{ "taskKeys": [ "T0", "T1" ] }] }

zooniverse / panoptes