search

oirlab / liger_iris_pipeline

Data Reduction System (DRS) for the Thirty Meter Telescope IRIS imager/spectrograph
https://oirlab.github.io/iris-pipeline
0 stars 3 forks source link

Minor refactoring of parse_subarray_map_step and test, not completed #69

Open astrobc1 opened 3 days ago

astrobc1 commented 3 days ago

@zonca Could you take a look at my progress on ParseSubarrayMapStep? It fails in this loop:

https://github.com/oirlab/liger_iris_pipeline/blob/d1cd12c53ee5ab2a216e0790d1bb59488f9a06ce/liger_iris_pipeline/parse_subarray_map/parse_subarray_map_step.py#L49-L50

result is a LigerIrisImage and result.meta has no attribute subarr_map to store a list of xstart, ystart, xsize, etc. The schemas are consistent with this. There is result.meta.subarray, but it is designed for one subarray; result.meta.subarray = each works for a single dict.

Can a single LigerIrisImage contain multiple subarrays at this stage? Eventually we should consider that subarrays are only relevant for IRIS, but maybe not just yet.

zonca commented 2 days ago

@astrobc1 reading https://oirlab.github.io/iris-pipeline/parse_subarray_map/index.html it seems like the purpose of the step is to create those fields.

we definitely had subarray map at:

https://github.com/oirlab/liger_iris_pipeline/blob/bce4533eb5dd0eb872e0b0394a78e1cbf0b24a67/iris_pipeline/datamodels/schemas/iris_image.schema.yaml#L17-L21

my bad, I lost that field in the refactoring, it defintely was there, see https://github.com/oirlab/liger_iris_pipeline/blob/bce4533eb5dd0eb872e0b0394a78e1cbf0b24a67/iris_pipeline/datamodels/schemas/tmt_core.schema.yaml#L1979-L1996

I now restored it.

zonca commented 2 days ago

I think we were able to have multiple subarrays, but I don't remember exactly how it worked, let me look into that and report back here.

zonca commented 2 days ago

@astrobc1 actually no, each file only has 1 subarray, see for example https://github.com/oirlab/liger_iris_pipeline/blob/master/liger_iris_pipeline/tests/test_merge_subarrays.ipynb

so I think subarray map is defined in the "Full frame" to keep track of the subarrays, but the subarrays are not actually in that file but in separate fits files.

zonca commented 2 days ago

example of subarray is at https://figshare.com/articles/dataset/TMT-IRIS_raw_science_frame_subarray/11953512

zonca commented 2 days ago

@astrobc1 there are some examples and explanation around page 60 of the design document