search

angelolab / ark-analysis

Integrated pipeline for multiplexed image analysis
https://ark-analysis.readthedocs.io/en/latest/
MIT License
71 stars 25 forks source link

Pixie percentile saving #1020

Closed camisowers closed 1 year ago

camisowers commented 1 year ago

If you haven't already, please read through our contributing guidelines before opening your PR

What is the purpose of this PR?

  1. Closes #1016. Implement intermediate saving of per image 99.9th percent values.

In addition, two small corrections were made to the pixie code.

  1. adds a new channel_percentile_postnorm argument to create_pixel_matrix(), rather than hard coding 0.999.
  2. add fix for the edge case where an image has no positive pixels (previously would error out) (@HPiyadasa ran into this issue when trying to pixels cluster only SMA+ pixels within a mask)
  3. Should also close #900.

How did you implement your changes

  1. Add image values to quantile_data.csv as they are processed. Also check the FOVs included in the file upon a restart, on the off chance a FOV had a feather file written, but the quantile data failed to be saved. Such images will be re-processed.
  2. Set default channel_percentile_postnorm=0.999.
  3. If there are no pixel cluster labels in an image, simply return an empty array rather than trying to concatenate an empty list (this causes an error).

Remaining issues Long term, we could probably prevent zero positive pixel images from attempting to be processed at all, but that likely requires a more in depth fix and the above solution works fine for now.

camisowers commented 1 year ago

Consulted with @alex-l-kong and removed a failing test that was resulting from the missing channel normalization code. Will get added back in #985.