hammerlab / cytokit

Microscopy Image Cytometry Toolkit
Apache License 2.0
115 stars 18 forks source link

Add Processing Step Statistics #6

Closed eric-czech closed 6 years ago

eric-czech commented 6 years ago

Each of deconvolution, drift compensation, and best focal plane selection result in some information that should be written into a file for QA. This should include at least:

Writing them all to a single processor_statistics.json file would probably make sense, as especially the best focal plane information is useful in other parts of the pipeline/analysis.

Ultimately, it would be great to turn anomalies in these numbers into errors, but given that it is a long running operation it may make more sense to simply have a "check_statistics" command or something of the like to point out noteworthy conditions and then let the user decide what to do about them.

nsamusik commented 6 years ago

good idea!

also, it would be great to identify empty tiles and remove them from processing...

also identify and remove planes with no in-focus objects, perhaps based on the focus score that you are getting during the BestFocus stage

On Thu, May 3, 2018 at 3:33 PM Eric Czech notifications@github.com wrote:

Each of deconvolution, drift compensation, and best focal plane selection result in some information that should be written into a file for QA. This should include at least:

  • Drift compensation translation vectors
  • Deconvolution mean intensity adjustment ratios
  • Deconvolution before and after SSIM or NRMSE? This may be a good way to get a sense of which tiles/cycles/channels changed the most from applying deconvolution
  • Focal plane classifications for each z-plane as well as the best z plane chosen

Writing them all to a single processor_statistics.json file would probably make sense, as especially the best focal plane information is useful in other parts of the pipeline/analysis.

Ultimately, it would be great to turn anomalies in these numbers into errors, but given that it is a long running operation it may make more sense to simply have a "check_statistics" command or something of the like to point out noteworthy conditions and then let the user decide what to do about them.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hammerlab/codex/issues/6, or mute the thread https://github.com/notifications/unsubscribe-auth/ADacL4aWzzTdOiUGX_nqnwEsvDSO_Jsrks5tu4WWgaJpZM4Tx4eA .

-- Nikolay


This message is intended for the named addressee(s) only. It may contain

privileged and confidential information and protected by a copyright. Any disclosure, copying or distribution of this message is prohibited and may be unlawful. If you are not the intended recipient, please destroy this message and notify me immediately. Thank you for your cooperation.

eric-czech commented 6 years ago

Makes sense @nsamusik , that would be great to have a good way to identify and remove/ignore those.