fjug / MoMA

MoMA - the MotherMachine Analyzer
4 stars 7 forks source link

Support single GL image stacks #2

Closed fjug closed 7 years ago

julou commented 8 years ago

in practice, it is convenient to save MoMA output along with the corresponding dataset. but for a project with a few 100s of GCs (typically 1000-2000 image file per GC), these data files are buried in a directory architecture that makes really long to traverse...

Currently >10' to find those data files in Matthias' project on our (slow - because GPFS?) cluster storage.

Not even talking of the nightmare of transferring such directories by sftp. It requires to tar.gz them first which can also take very long…

julou commented 8 years ago

Florian's recent comment on that in an email:

Let me start with a few more words about tif saving/loading: I agree, it is indeed frustrating. One would think that loading and saving are the two MOST basic functions that should just work reliably. Unfortunately the opposite is the case and the mix of IJ1 and new IJ2/BioFormats is a pain. While this is quite well ‘hidden’ for users, as soon as you use these things programmatically matters only get worse. MoMA uses quite some IJ2 features that conflict with the fast IJ1 stack and hyperstack saving routines. While I cannot work around the necessary IJ2 dependencies, I am required to work my way around the mess. Luckily it really seems as if practical solutions are around the corner. Just last week I had intense discussions with BioFormats and Scifio developers about these issues. In fact, I need a (stable) solution for robust stack manipulations and tiff saving/loading also for other projects I am working on...

My answer was that I'm afraid that "around the corner" might still mean 10-20 months before a stable version is usable; and we've been asking for this for over a year now…

julou commented 8 years ago

btw whenever this is implemented, it would be nice to change MoMA's arguments so that the number of channels doesn't need to be given on the command line anymore. Instead, it would be useful to have a parameter that sets which channel is to be used for the segmentation (e.g. channel 2 if phase contrast is the second channel in the stack).

fjug commented 7 years ago

@julou we are currently implementing this feature. The following question arose: Should the default behavior of mmpreprocess be so save stacks or, as it is for now, to save image sequences. Saving stacks per default would break backward compatibility, but maybe it is what you want. In any case, the non-default can always be chosen by an additional command line flag. Your call! :)

julou commented 7 years ago

if we're the only ones concerned with backward compatibility, I wouldn't mind changing the default. Because I think it makes data transfer / copy / backup so much easier…

I imagine that it would still be possible to get the sequences behaviour using an argument, or?

fjug commented 7 years ago

Yes, a simple argument makes it behave as it did so far. Thanks for you opinion. 👍

julou commented 7 years ago

available as of MoMA workshop