Saving process

[vigji]

Moving here code for the saving process from sashimi, and adding testing and cleanup in the process. Changes so far:

• moved email notification out of this process to separate module;

[codecov[bot]]

Codecov Report

Merging #8 into master will not change coverage. The diff coverage is 0.00%.

@@           Coverage Diff           @@
##           master      #8    +/-   ##
=======================================
  Coverage    0.00%   0.00%            
=======================================
  Files          24      26     +2     
  Lines        3179    3308   +129     
=======================================
- Misses       3179    3308   +129     

Impacted Files	Coverage Δ
scopecuisine/__init__.py	0.00% <0.00%> (ø)
scopecuisine/email_notification.py	0.00% <0.00%> (ø)
scopecuisine/saving_process.py	0.00% <0.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update d4c7ace...2decb12. Read the comment docs.

[vilim]

To add Zarr and TIFF backends, I think it would make sense to add an extra (abstract) class: DataDestination, which would have a preallocate, add_plane, add_volume and finalize methods. We only need to think a bit about how to handle both 2p, volumentric and planar lightsheet cases (planar lightsheet probably just as 1-z-slice volumes)

[vigji]

Make sense, and planar lightsheet should definitively be just a 1z-slice volume.

[vigji]

@vilim following the current flow of the code, the saver should be initialised agnostic to xy plane size and then inferred it from the first stack. I think it would be cleaner to calculate them continuously with the (unsaved) frames that are streamed to the StackSaver and instantiate the DataDestination instance already with the proper size (in which case, the split dataset format could become something like a multiple inheritance DataDestination + SplitDataset object with methods for saving)

[vilim]

Well, the thing is that if the experiments gets manually stopped beforehand, you still want to save a vaild dataset, so the full dimensions cannot be know beforehand. I think it's fine that the DataDestination initializes the temporary storage (so the saver doesn't do that at all)

[vigji]

So in principle the full size should be known beforehand and "preallocated" (in the SplitDs attributes), but then the case of interruption should be handled?

[vilim]

This is one variant, the other is just to be agnostic to it. I don't care which it is as long as it works in all cases.

[vilim]

Note: I have added a notifier interface to master, so this needs to be changed a bit. (maybe adding Diego's picture-sending functionality from the 2p, and then making 2p rely on this, but that would be a separate PR).