search

vanvalenlab / deepcell-tf

Deep Learning Library for Single Cell Analysis
https://deepcell.readthedocs.io
Other
411 stars 94 forks source link

Mesemr reproducibility #611

Open azulaynofar opened 2 years ago

azulaynofar commented 2 years ago

Describe the bug I was using Mesmer (through ark-analysis repository) roughly two months ago to segment my images and I was super happy with the results. This week, when I segmented additional images, it seemed like I was getting malformed segmented cells, so I re-run the same images I segmented 2 months ago and indeed I see a huge difference. I'm pointing out that the cell type I'm trying to segment was probably not trained by Mesmer (goblet cells), but still, I managed to get it segmented by using a cytoplasmatic marker and using a smaller scaling factor (since these cells are larger than other cells). I understood from a similar issue opened here that you re-train the network using a different cleaned-up method? I was wondering if there is backward compatibility to the older Mesmer version that can be used via the ark-analysis repo?

To Reproduce Steps to reproduce the behavior:

  1. Run segmentation template using ark-analysis repo (mucin - nuclear, keratin + NA-K + E-cad - membranal)
  2. See different results in segmentation

Expected behavior same segmentation mask

Screenshots image

Desktop (please complete the following information):

ngreenwald commented 2 years ago

Hey Nofar, We updated the model last night with additional changes. Can you try re-running again to see if that solved the issue?

azulaynofar commented 2 years ago

Hi Noah, thank you for the quick reply. I checked the new model and though it improved the results I still think the older model performed better for my task. how could I still use the older model through ark? in general I think it would be super helpful to keep backward compatibility with ark once you update deepcell (in case it is possible, if not - keep a log file with the model version)

ngreenwald commented 2 years ago

Can you post a couple images of both versions? I did a pretty thorough comparison and thought the new version in aggregate was better.

There's currently not a way to access old versions of the model through ark, since it relies on the server to process the images through the deployed model. If you want to use older versions, you'll need to install the deepcell-tf repo on your own GPU and run it there, as it has access to all of the previous models.