search

c3-time-domain / SeeChange

A time-domain data reduction pipeline (e.g., for handling images->lightcurves) for surveys like DECam and LS4
BSD 3-Clause "New" or "Revised" License
0 stars 4 forks source link

Background object #205

Closed guynir42 closed 3 months ago

guynir42 commented 7 months ago

The thing we need is a background object to go along with the Image object. This allows us to do a few things: 1) Hold on to different types of background data, including a map the same size as the image, a 2D polynomial where we only keep the coefficients, a constant value, or even "zero" (which means the background was already subtracted for this Image). 2) We can save to database and disk the results separately from the Image. 3) We can use Provenance to track versions of the background subtraction code/parameters. 4) We can lazy load the background data whenever we want to get the background subtracted image.

There are some problems with this model: a) If the background is a scalar or zero, would we still save a file? If not, how to square this with the FileOnDiskMixin class requiring a filepath. b) Where exactly does the background calculation happen? Is it in preprocessing, in extraction, or in a separate process in between those two? c) Do we add another process with it's own provenance? Or should we append the background as a secondary product of one of the existing processes (preprocessing/extraction)?

I think it will be necessary to save the parameter choices for background estimation, because at some point we will need to allow different, competing algorithms to run.

However, I am not sure we can separate this from the source extraction. In some cases (e.g., in dense fields where this is the most critical) the background estimation MUST happen along with source extraction, e.g., when using scene modeling.

I suggest we append the Background product to the extraction process. It already creates two objects that share the same Provenance "process", which are SourceList and PSF. The advantage is that we don't need to add another step in the pipeline (that would also require lots of refactoring) but only add an output object. The downside is that if you change the background subtraction code you must also re-do extraction and PSF calculation (and vice versa).

guynir42 commented 6 months ago

Could be a good second issue or third issue.

guynir42 commented 3 months ago

Addressed by #308