search

gerlichlab / looptrace

Fork (from EMBL Gitlab) of the looptrace project: https://git.embl.de/grp-ellenberg/looptrace
MIT License
2 stars 1 forks source link

Separate extraction of background data #322

Closed vreuter closed 4 months ago

vreuter commented 4 months ago

keep the construction of the NPZ exactly as-is, but--when the config specifies to subtract background--extract a parallel structure with just the background data. then, when doing the tracing, use the presence or absence of that additional data stack to determine what to do (subtract background or not). don't store the background-subtracted arrays as NPY/NPZ, but do store them as ZARR, basically replacing the ZARRs that currently wind up in locus_spots_visualisation subfolder

i think this has a few desirable properties: a. consistency : it keeps the NPZ as-is, and identical whether background is being subtracted or not b. modularity : data are compartmentalised by function (background vs. rest (modulo the fact that in the absence of locusGrouping, what's potentially the background timepoint will already be in the NPZ stacks) c. minimality : this would add a relatively small, simple storage footprint (the NPZ for background, so a 12x24x24 volume for each trace) d. removes redundancy: currently the ZARR in locus_spots_visualisation is entirely redundant w/ the NPZ, just in a different format e. information content: both the non-background-subtracted (as NPZ) and background-subtracted (as ZARR) data will be readily available after the pipeline completes

vreuter commented 4 months ago

Related to #237 , as we've been extracting all timepoints' data