AVHRR CLASS Filename prefix prevents reading with Scene.

[jeanlucshaw]

Bug description I am setting up a processing chain for NOAA and MetOp satellite data acquired daily through a subscription with CLASS. These files arrive with a numeric prefix to the typical filename for AVHRR binary data. Satpy's Scene class can read these file if I rename them to remove the numeric prefix, but not if I leave it in place. The error message is: No supported files found.

To Reproduce

 import satpy

 # Same for both files
 reader_kwargs = {'tle_dir': '../tle', 'tle_name': 'TLE_NOAA19.txt'}

 # The file I get through subscriptions to CLASS (produces the error message)
 filename = ['9304405277.NSS.HRPT.NP.D22209.S0011.E0023.B6941818.MI']
 scene_a = satpy.Scene(filenames=filename,
                       reader='avhrr_l1b_gaclac',
                       reader_kwargs=reader_kwargs)

# The same file copied to a name without the numeric prefix (no error message)
filename = ['NSS.HRPT.NP.D22209.S0011.E0023.B6941818.MI']
scene_b = satpy.Scene(filenames=filename,
                      reader='avhrr_l1b_gaclac',
                      reader_kwargs=reader_kwargs)    

Expected behavior I would expect Scene to either recognize that this is supported file regardless of file name changes, warn that the file name is unexpected but read anyway, or produce an error message specifying that the file name needs to be changed.

Actual results [DEBUG: 2022-08-01 08:26:57 : satpy.readers.yaml_reader] Reading ('/opt/anaconda3/envs/py37/lib/python3.7/site-packages/satpy/etc/readers/avhrr_l1b_gaclac.yaml',) [WARNING: 2022-08-01 08:26:57 : satpy.readers.yaml_reader] No filenames found for reader: avhrr_l1b_gaclac [WARNING: 2022-08-01 08:26:57 : satpy.readers] Don't know how to open the following files {'9304405277.NSS.HRPT.NP.D22209.S0011.E0023.B6941818.MI'}

Screenshots

Environment Info:

• OS: Linux SMP Debian 4.9.320-2 (2022-06-30)
• Satpy Version: 0.36.0
• PyResample Version: 1.23.0

[pnuu]

You can add your own filename pattern for these files by:

• create a base directory, e.g. $HOME/satpy_config
• create a config directory for the readers $HOME/satpy_config/readers
• copy https://github.com/pytroll/satpy/blob/main/satpy/etc/readers/avhrr_l1b_gaclac.yaml to $HOME/satpy_config/readers/
• at the end of the file, add a new entry to the file_patterns list, which is identical to the existing, but having an additional field '{numeric_prefix}.' in it
• set a environment variable SATPY_CONFIG_PATH=$HOME/satpy_config in you .bashrc or setting os.environ['SATPY_CONFIG_PATH'] = ... at the beginning of your script
• run your script

The other choice is to create a pull-request adding the new pattern to Satpy, but the above will get you ahead faster. The PR is naturally still wellcome :-)

[jeanlucshaw]

Thanks for writing back so fast @pnuu ! I am writing tools that will be used by a other people too, so if it could be built in to satpy, that would be convenient. I would be happy to tackle this myself. I have read the contributor guidelines and have satpy-dev set up. Can you tell me what the next steps would be?

[mraspaud]

@jeanlucshaw I agree that this can be added to satpy. The only thing you have to do I think is to add an item to the file pattern list here: https://github.com/pytroll/satpy/blob/main/satpy/etc/readers/avhrr_l1b_gaclac.yaml#L185 and then submit a PR. Good luck!

[jeanlucshaw]

OK I am on it. Should I wait until the pull request is done or close the issue now?

[mraspaud]

You can wait and close the issue through the PR (the line Fixes #xxx in the PR template takes care of that).

[jeanlucshaw]

I am getting a permission error when I try to push my branch, i.e.,

$ git push origin my-branch

As a new contributor, do I need to request an access token?

[gerritholl]

You need to fork the repo so that origin points to jeanlucshaw/satpy, then push there. Then set upstream to pytroll/satpy. There are user guides for this on GitHub and elsewhere on the web.

You do not need any access token to make a pull request based on a branch in your own fork.

[jeanlucshaw]

Fixed by PR #2157 .