search

spacetelescope / romancal

Python library to process science observations from the Nancy Grace Roman Space Telescope
https://roman-pipeline.readthedocs.io/en/latest/
Other
28 stars 28 forks source link

Make sure logs get included in L3 products. #1207

Closed schlafly closed 2 months ago

schlafly commented 2 months ago

This PR leads HLP products to have their "cal_logs" attribute filled out.

Previously these were not being populated because stpipe.core was explicitly checking that the output datamodel was an ImageModel. This PR changes that to check if it's either an ImageModel or a MosaicModel. It also clears the CalLogs when making a new output model to avoid some initial filler values.

Finally, a big portion of the HLP is processing the input L2s; flux calibration, sky matching, outlier detection. Those logs go into the L2 products which we do not save. This adds a new individual_image_cal_logs extension parallel to cal_logs that houses the individual L2 log messages. We probably want that?

Checklist

codecov[bot] commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 79.03%. Comparing base (1898ca5) to head (3d7d787). Report is 8 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1207 +/- ## ========================================== + Coverage 76.78% 79.03% +2.25% ========================================== Files 113 116 +3 Lines 7933 7985 +52 ========================================== + Hits 6091 6311 +220 + Misses 1842 1674 -168 ``` | [Flag](https://app.codecov.io/gh/spacetelescope/romancal/pull/1207/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope) | Coverage Δ | | *Carryforward flag | |---|---|---|---| | [nightly](https://app.codecov.io/gh/spacetelescope/romancal/pull/1207/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope) | `62.78% <ø> (?)` | | Carriedforward from [47b29e2](https://app.codecov.io/gh/spacetelescope/romancal/commit/47b29e2da0d706fe65edb25c63f0be4fd888a930?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope) | *This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

schlafly commented 2 months ago

Regtests are "passing"; two new failures because of the added individual_image_cal_logs bit in the asdf file. If we like this approach, we should exclude those from compare_asdf.

schlafly commented 2 months ago

After adding individual_image_cal_logs to the list of extensions to ignore, regtests actually pass. https://plwishmaster.stsci.edu:8081/job/RT/job/Roman-Developers-Pull-Requests/726/