[BUG]: AttributeError: lv2 'NoneType' object has no attribute 'lower'

[wjcramer]

Hi,

I'm very excited to use this pipeline for some new MIRI observations I'm working with of a nearby galaxy, it seems like such an incredible resource for the community! I've gotten through the lvl1 processing, but when I get to level 2, I get the following error:

[2024-01-30 14:13:54,639] INFO - Beginning lv2 for F560W [2024-01-30 14:13:54,639] INFO - Building asn files Traceback (most recent call last): File "/Users/wcramer2/Desktop/pjpipe/run_reprocessing.py", line 28, in pjp.do_pipeline() File "/Users/wcramer2/miniconda3/envs/jwst/lib/python3.11/site-packages/pjpipe/pipeline.py", line 513, in do_pipeline step_result = lv2.do_step() ^^^^^^^^^^^^^ File "/Users/wcramer2/miniconda3/envs/jwst/lib/python3.11/site-packages/pjpipe/lv2/lv2_step.py", line 127, in do_step asn_files = self.create_asn_files( ^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wcramer2/miniconda3/envs/jwst/lib/python3.11/site-packages/pjpipe/lv2/lv2_step.py", line 172, in create_asn_files tab = get_obs_table( ^^^^^^^^^^^^^^ File "/Users/wcramer2/miniconda3/envs/jwst/lib/python3.11/site-packages/pjpipe/utils/utils.py", line 480, in get_obs_table parse_fits_to_table( File "/Users/wcramer2/miniconda3/envs/jwst/lib/python3.11/site-packages/pjpipe/utils/utils.py", line 530, in parse_fits_to_table obs_label = im.meta.observation.observation_label.lower() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'lower'

The only key word I have set for lvl2 is: bgr_check_type = 'check_in_name'

I was wondering if what might be contributing is we don't have any "background" exposures for our data set. We have a particular image of a more blank part of the sky as part of the mosaic we were planning to use for a background image, but all images in the data set are currently labeled as "SCI" in the header. If this is causing it, any way to point to these particular frames to use for background?

[thomaswilliamsastro]

hmm, I think this may be more due to the metadata in your images. Can you loop over the images and let me know what this does (just in some new script)

from stdatamodels.jwst import datamodels

for file in files:
    with datamodels.open(file) as im:
        print(im.meta.observation.observation_label)

I suspect that there might be no observation label in your dataset, in which case this probably goes None and then .lower() will crash out. Haven't seen this before!

[low-sky]

I've encountered this with my Cy2 data. I just replaced these lines with

-        obs_label = im.meta.observation.observation_label.lower()
+        if im.meta.observation.observation_label is not None:
+            obs_label = im.meta.observation.observation_label.lower()
+        else:
+            obs_label = ''

[thomaswilliamsastro]

@low-sky this is what I was thinking would fix it too. I'll get a PR in tomorrow morning if you're happy to be a guinea pig testing @wjcramer ?

[wjcramer]

Certainly happy to test anything out! When I print(im.meta.observation.observation_label) I do get 'None' for all frames. I tried manually editing the utils.py file in my installed version of the pjpipe package with Erik's fix and it worked!

[wjcramer]

I was wondering if you can comment on what lv2 does if there are no background images specified. If I want to use a frame from the corner of the mosaic I expect to be most empty for background estimates for example, can I put something like this into the config file?

bgr_check_type = 'check_in_name' background_name = 'mosaic_frame_22.fits'

[thomaswilliamsastro]

@wjcramer I've just pushed a branch that should fix the crash. Try

pip3 install git+https://github.com/PhangsTeam/pjpipe.git@obs-name-fix

and let me know if that fixes things. In terms of specifying a certain observation for a background, we currently don't have that functionality. The options we have are "check_in_name" where it'll check against a match in im.meta.target.proposer_name, and "parallel_off" where it will look at whether the observations were taken in parallel with other science. We could get it to look for filenames if you don't specifically have anything flagged for background, but that would be another feature request :)