Add ability to run SLAPP transform_pipeline on production data

[njmei]

• [ ] Squash commits once PR looks good

Purpose

• Enables running transform pipeline from a file input (rather than dev postgres DB)
• Enables output of transform pipeline to a file (rather than dev postgres DB)

Input Format

A typical example:

{
  "experiment_id": "1000744365",
  "local_to_global_roi_id_map": {
    "73": 2000159,
    "8": 2000255,
    "113": 2000308
  },
  "binarized_rois_path": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/binarize_output.json",
  "traces_h5_path": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/roi_traces.h5",
  "movie_path": "/allen/programs/braintv/production/visualbehavior/prod3/specimen_935565389/ophys_session_1000439105/ophys_experiment_1000744365/processed//motion_corrected_video.h5"
}

Output (a slapp manifest)

{"experiment-id": 1000744365, "roi-id": 2000255, "source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/outline_2000255.png", "roi-mask-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/mask_2000255.png", "full-video-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/full_video.webm", "video-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/video_2000255.webm", "max-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/max_2000255.png", "avg-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/avg_2000255.png", "trace-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/trace_2000255.json", "full-outline-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/full_outline_2000255.png"}
{"experiment-id": 1000744365, "roi-id": 2000159, "source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/outline_2000159.png", "roi-mask-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/mask_2000159.png", "full-video-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/full_video.webm", "video-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/video_2000159.webm", "max-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/max_2000159.png", "avg-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/avg_2000159.png", "trace-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/trace_2000159.json", "full-outline-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/full_outline_2000159.png"}
{"experiment-id": 1000744365, "roi-id": 2000308, "source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/outline_2000308.png", "roi-mask-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/mask_2000308.png", "full-video-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/full_video.webm", "video-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/video_2000308.webm", "max-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/max_2000308.png", "avg-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/avg_2000308.png", "trace-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/trace_2000308.json", "full-outline-source-ref": "/allen/aibs/informatics/danielk/dev_LIMS/supplemental/1000744365/slapp_artifacts/20200908152036/full_outline_2000308.png"}

[codecov-commenter]

Codecov Report

Merging #125 into master will decrease coverage by 0.44%. The diff coverage is 89.53%.

@@            Coverage Diff             @@
##           master     #125      +/-   ##
==========================================
- Coverage   92.96%   92.52%   -0.45%     
==========================================
  Files          15       15              
  Lines         867      936      +69     
==========================================
+ Hits          806      866      +60     
- Misses         61       70       +9     

Impacted Files	Coverage Δ
slapp/transfers/utils.py	98.95% <80.00%> (-1.05%)	:arrow_down:
slapp/utils/merge_utils.py	95.00% <80.00%> (-0.84%)	:arrow_down:
slapp/transforms/transform_pipeline.py	91.20% <90.76%> (-1.10%)	:arrow_down:
slapp/rois.py	93.44% <90.90%> (-0.79%)	:arrow_down: