fkirmani / diffimageml

scrum sprint for ML transient detection/classification in difference images
BSD 3-Clause "New" or "Revised" License
2 stars 4 forks source link

Sketch a plan for scaling single file operations to databases or large data repositories #21

Closed AdkPete closed 3 years ago

srodney commented 3 years ago

This feels a little insufficient, but for the moment here's a sketch:

data_transfer_plan

What we're missing here is any real thinking about a database to track the planting of fakes over multiple epochs, but that seems to be a largely separate issue from dealing with a remote data repository.

srodney commented 3 years ago

This is the sketch for how we could operate on a collection of images in a time series, using simulated SN so that the brightness of a source evolves over time. This whole flowchart fits inside the "Fake Planting" box inside the yellow box in the data-transfer plan above. Building this functionality is NOT a target for the current Sprint2, but is part of the larger product plan.

fakeSN_catalog_plan