In my previous PRs, I laid the groundwork for supporting general (non-SDSS) datasets, by making an interface where Celeste creates a SurveyDataSet object describing how the data is stored, and then calling load_images(dataset, box) and expecting to get back an array of survey-agnostic Celeste.Image objects containing image data. The SDSS data I/O routines now conform to this interface.
This PR lays out implementing load_images for a new SurveyDataSet type, called DECALSDataSet, which represents data from the Dark Energy Camera Legacy Survey. It goes as far as determining which images overlap the given bounding box, but does not yet load them.
The remaining to-dos to actually load the images are fairly straight-forward, but I don't know when I'm going to have time to implement them so I'm handing this off now. Most of the hard part is figuring out where the raw and calibrated images are actually located.
Here are the remaining to-do items (also commented in the code itself):
Create a function to read in both raw and calibrated images and determine calibration array (simply divide the calibrated image by the raw image)
Change the Celeste.Image type to accept a 2-d rather than 1-d calibration array (in general, calibration will not be constant in one dimension like in SDSS.)
Add a has_background boolean flag to Celeste.Image (e.g., make the background array optional). Then after calling load_images(dataset, box), check has_background and if false, calculate the background with SEP (and set flag to true). This allows us to optionally read the background from the dataset (e.g., SDSS supplies one, but DECaLS does not).
Add option to SDSSDataSet type to read background or not (e.g., use the SDSS-calculated background or let Celeste determine the background).
Add general config options for background subtraction, detection parameters (OK, not actually necessary for load_images, but nice to have).
Do something for the PSF (needs investigation into DECaLS PSF model). OK, this is another hard part.
Testing: make smaller versions of some calibrated and raw images images in that ra, dec range (remove ccds from files) and put them at NERSC for download with a Makefile (Are the raw and calibrated images actually on disk at NERSC?)
In my previous PRs, I laid the groundwork for supporting general (non-SDSS) datasets, by making an interface where Celeste creates a
SurveyDataSet
object describing how the data is stored, and then callingload_images(dataset, box)
and expecting to get back an array of survey-agnosticCeleste.Image
objects containing image data. The SDSS data I/O routines now conform to this interface.This PR lays out implementing
load_images
for a newSurveyDataSet
type, calledDECALSDataSet
, which represents data from the Dark Energy Camera Legacy Survey. It goes as far as determining which images overlap the given bounding box, but does not yet load them.The remaining to-dos to actually load the images are fairly straight-forward, but I don't know when I'm going to have time to implement them so I'm handing this off now. Most of the hard part is figuring out where the raw and calibrated images are actually located.
Here are the remaining to-do items (also commented in the code itself):
Create a function to read in both raw and calibrated images and determine calibration array (simply divide the calibrated image by the raw image)
Change the
Celeste.Image
type to accept a 2-d rather than 1-d calibration array (in general, calibration will not be constant in one dimension like in SDSS.)Add a
has_background
boolean flag toCeleste.Image
(e.g., make the background array optional). Then after callingload_images(dataset, box)
, checkhas_background
and if false, calculate the background with SEP (and set flag totrue
). This allows us to optionally read the background from the dataset (e.g., SDSS supplies one, but DECaLS does not).Add option to SDSSDataSet type to read background or not (e.g., use the SDSS-calculated background or let Celeste determine the background).
Add general config options for background subtraction, detection parameters (OK, not actually necessary for
load_images
, but nice to have).Do something for the PSF (needs investigation into DECaLS PSF model). OK, this is another hard part.
Testing: make smaller versions of some calibrated and raw images images in that ra, dec range (remove ccds from files) and put them at NERSC for download with a Makefile (Are the raw and calibrated images actually on disk at NERSC?)