pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.07k stars 295 forks source link

Tropomi l2 reader needs to handle more filenames #1077

Closed TAlonglong closed 4 years ago

TAlonglong commented 4 years ago

As the tropomi l2 reader is now it handles: {platform_shortname:3s}_{data_type:4s}_{level:3s}_{product:3s}____{start_time:%Y%m%dT%H%M%S}_{end_time:%Y%m%dT%H%M%S}_{orbit:5d}_{collection:2d}_{processor_version:6d}_{creation_time:%Y%m%dT%H%M%S}.nc

eg S5P_OFFL_L2NO2__20180709T170334_20180709T184504_03821_01_010002_20180715T184729.nc

But there are other products than NO2, eg AER_AI S5P_NRTI_L2__AER_AI_20200220T163359_20200220T163859_12205_01_010302_20200220T170635.nc

The documentation I have found http://www.tropomi.eu/sites/default/files/files/S5P-KNMI-L2-0026-MA-Product_user_manual_for_the_Sentinel_5_precursor_Aerosol_Index_product-1.0.0-20180613_signed.pdf

See table 1 and chapter 4.1

all available products are listed.

So I suggest to somehow alter the {product:3s} to be more general to handle all types of products.

I think the SEVIRI HRIT have something like this, so for Tropomi this will be. {platform_shortname:3s}_{data_type:4s}_{level:3s}_{product:_<6s}_{start_time:%Y%m%dT%H%M%S}_{end_time:%Y%m%dT%H%M%S}_{orbit:5d}_{collection:2d}_{processor_version:6d}_{creation_time:%Y%m%dT%H%M%S}.nc

djhoese commented 4 years ago

Gross. I've never seen a filenaming scheme like that. I think if the 3 is removed from the product parameter and the ____ (for underscores) are replaced with _ (one underscore) then it should work. @TAlonglong Do you think you could make a pull request for this and test it out?

CC @tommyjasmin

tommyjasmin commented 4 years ago

Dave - this is not as gross as appears and it's great @TAlonglong noticed this. The reason for the underscore padding simply to ensure a consistent filename length is maintained in the context of atmospheric chemistry product names with varying lengths, e.g. :

I'm surprised this was still in the code, I thought we'd fixed this and I see we did correct this in the McIDAS-V codebase.