DES-SL / Y6_Bulk_Coadd_Cutouts

Codebase for large-scale cutout production from DES co-added images on the FNAL cluster
MIT License
0 stars 2 forks source link

Adding stack2image conversion script #21

Open kadrlica opened 3 years ago

kadrlica commented 3 years ago

@rmorgan10 @erikzaborowski This is a first pass at the stack2image conversion script. Creates image HDUs for image, psf, and mask for each band. Pulls image header information from the external and built-in metadata. Should be pretty memory efficient, but it is not optimized for speed (1000 cutouts in 70s).

This is probably not ready to merge until we can run on some of the DELVE cutouts where we have user requests.

rmorgan10 commented 3 years ago

I think we should also add a "test_stack2image.py" script to the test/ directory before we merge, so any tests we do for user requests can certainly make their way into that script as unit tests.

I'll look at the functionalities here more in depth before our meeting tomorrow

kadrlica commented 3 years ago

Do you have a small data set for running tests?

rmorgan10 commented 3 years ago

Yeah it's gitignored but it's on the des machines. The way the test_cutout.py is set up, it just works on 10 objects. Do less test/test_data/testtile.tab.gz to see the COADD ID, RA, and DEC for those 10. I think there's a README in that directory (on the des machines) that explains the process

rmorgan10 commented 3 years ago
[rmorgan@des50 test_data]$ pwd
/data/des81.b/data/stronglens/PRODUCTION/Y6_CUTOUT_IMAGES/Y6_Bulk_Coadd_Cutouts/test/test_data
[rmorgan@des50 test_data]$ cat README 
Compiled by Robert Morgan, Jan 6, 2021.

Information
----------------
* testtile is tile DES2228+0001, JOBNUM 4575p01.
* files in multiple_files/ are empty.
* testtile.tab.gz has 10 objects' RA, DEC, and COADD_ID from DES2228+0001.

Directory Contents
----------------------
.
├── multiple_tiles
│   ├── multiple_tiles_r1_g.fits.fz
│   └── multiple_tiles_r2_g.fits.fz
├── README
├── testtile
│   ├── testtile_r123_g.fits.fz
│   ├── testtile_r123_g_psfcat.psf
│   ├── testtile_r123_i.fits.fz
│   ├── testtile_r123_i_psfcat.psf
│   ├── testtile_r123_r.fits.fz
│   ├── testtile_r123_r_psfcat.psf
│   ├── testtile_r123_Y.fits.fz
│   ├── testtile_r123_Y_psfcat.psf
│   ├── testtile_r123_z.fits.fz
│   └── testtile_r123_z_psfcat.psf
└── testtile.tab.gz
kadrlica commented 3 years ago

@rmorgan10 I've looked at these test files. Do you have output metadata somewhere?

rmorgan10 commented 3 years ago

The only metadata columns I have in the tests are RA, DEC, and COADD_OBJECT_ID in the testtile.tab.gz

If you want, I can grab the other metadata columns for the 10 objects in that file and add them to testtile.tab.gz to better simulate the real metadata

kadrlica commented 3 years ago

Sure, might as well have a test with some more complexity.

rmorgan10 commented 3 years ago

I've added the other metadata columns to testtile.tab.gz