LIMS work for new Feature Extraction and Inference workflow step

[pickles-bread-and-butter]

Overview:

Introducing a new workflow step for Feature Extraction and Classification requires changes to the LIMS strategies, job transitions, and updates to LIMS tables to integrate with the existing pieces of the Ophys Segmentation Pipeline.

Tasks:

• [x] Create a new LIMS strategy for running the Feature Extraction and Inference Workflow Step
• [x] Create new LIMS job queue record in the database (LIMS datafix script)
• [x] Create new LIMS executable record in the database(LIMS datafix script)
• [x] Add new columns to LIMS ophys_cell_segmentation_runs table (db migration script). @isaak-willett What new columns?
• [x] Update the LIMS "roi_exclusion_labels" table. @isaak-willett I don't see this table in LIMS. Please clarify.

Acceptance Criteria:

• [x] On a test LIMS server LIMS successfully transitions from Ophys_extract_traces workflow step to Feature_extraction_and_inference workflow step, executes all steps in Feature_Extraction_and_Inference workflow step successfully, and writes the ROIs to LIMs with the correct exclusion labels.

Note

Please tag any LIMS check-ins with PBS-2374 http://jira.corp.alleninstitute.org/browse/PBS-2374

[njmei]

Helpful resource for setting up a local dev version of LIMS:

http://confluence.corp.alleninstitute.org/pages/viewpage.action?pageId=60855687

[njmei]

Added some additional necessary compatibility work on modules:

https://github.com/AllenInstitute/ophys_etl_pipelines/pull/27

[njmei]

@wbwakeman Since the Singularity images have not been nailed down and neither has a classifier model, it's not really possible to satisfy the acceptance criteria at this point in time. Should I just keep this issue open then or should the more holistic validation criteria described in this issue be put in a separate issue?

At least for the OphysClassifyRois LIMS strategy, things look to 'work' by using job_test_runs and creating synthetic input and output files.

[wbwakeman]

Not sure I understand. #10 is marked as "Done". So, why are we not able to run it?

[njmei]

https://files.slack.com/files-pri/TLEGP6YLA-F018S9D329Z/image.png