Open schlafly opened 2 months ago
While checking a new version of the release I did the following as a kind of SDP-lite set of invocations:
# make level "2a" files
strun roman_elp r0000101001001001001_01101_0001_WFI01_uncal.asdf --steps.tweakreg.skip=True --suffix=sourcedetection
strun roman_elp r0000101001001001001_01101_0002_WFI01_uncal.asdf --steps.tweakreg.skip=True --suffix=sourcedetection
# make association file linking these
asn_from_list *_WFI01_sourcedetection.asdf --product-name aoeu -o asn_sourcedetection.json
# make final cal files
strun romancal.step.TweakRegStep asn_sourcedetection.json --suffix=cal
# make association file for cal files
asn_from_list *_cal.asdf --product-name=r0000101001001001001_F158_visit_r274dp63x32y80 -o mosaic.json
# this file then needs to get hand edited to add the target skycell name to the "target" bit of the asn file
# i.e., add target="r274dp63x32y80" to the json file
# make final mosaic product
strun roman_mos mosaic.json
This makes the following files:
asn_sourcedetection.json # association file we made for source detection files, could be ignored?
mosaic.json # association file we made for cal files, could be ignored?
r0000101001001001001_01101_0001_WFI01_cal.asdf # L2 file
r0000101001001001001_01101_0001_WFI01_cat.asdf # corresponding cat file
r0000101001001001001_01101_0001_WFI01_segm.asdf # corresponding segmentation file
r0000101001001001001_01101_0001_WFI01_sourcedetection.asdf # "2a" file that we shouldn't save
r0000101001001001001_01101_0001_WFI01_uncal.asdf # uncal input file
r0000101001001001001_01101_0002_WFI01_cal.asdf # L2 file
r0000101001001001001_01101_0002_WFI01_cat.asdf # corresponding cat file
r0000101001001001001_01101_0002_WFI01_segm.asdf # corresponding segm file
r0000101001001001001_01101_0002_WFI01_sourcedetection.asdf # corresponding sourcedetection file
r0000101001001001001_01101_0002_WFI01_uncal.asdf # uncal input file
r0000101001001001001_F158_visit_r274dp63x32y80_cat.asdf # L3 catalog
r0000101001001001001_F158_visit_r274dp63x32y80_i2d.asdf # L3 image
r0000101001001001001_F158_visit_r274dp63x32y80_segm.asdf # L3 segmentation map
skycell_wcs.asdf # temporarry file that should be deleted
I think something of that form would exercise much of the needed SDP interface, but @ctbheller , let me know if there's something else I can add that would be helpful.
Updating file names and adding the --target argument:
# make level "2a" files
strun roman_elp r0000101001001001001_0001_wfi01_uncal.asdf --steps.tweakreg.skip=True --suffix=sourcedetection
strun roman_elp r0000101001001001001_0001_wfi02_uncal.asdf --steps.tweakreg.skip=True --suffix=sourcedetection
# make association file linking these
asn_from_list *_sourcedetection.asdf --product-name aoeu -o asn_sourcedetection.json
# make final cal files
strun romancal.step.TweakRegStep asn_sourcedetection.json --suffix=cal
# make association file for cal files
asn_from_list *_cal.asdf --product-name=r0000101001001001001_F158_visit_r274dp63x32y80 -o mosaic.json --target r274dp63x32y80
# make final mosaic product
strun roman_mos mosaic.json
works for the proposed intermediate release.
In order to check out the full SDP interface, we could make a new regtest / smoketest that runs all of the ELP / HLP steps.
This is annoying in that it would be slow. We would want to do something like the following:
It's not obvious to me that I would want to do very much with the resulting files (e.g., compare_asdf, etc.); I might continue to leave that to the existing set of regtests. But it's at least important to know that all of those steps run and to have a place with a SDP-lite pipeline that the real SDP pipeline can emulate. This would make us more aware when the SDP commands need to change slightly to accommodate changes to the pipeline.