pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.06k stars 294 forks source link

Missing the `time` dimension in the tropomi_l2 file generated by CF writer #1947

Open zxdawn opened 2 years ago

zxdawn commented 2 years ago

Describe the bug

In the original TROPOMI L2 NO2 file, GEO2D variables always have the time dimension: image

It is useful for open_mfdatasets():

xr.open_mfdataset(['./201908/S5P_PAL__L2__NO2____20190811T004435_20190811T022605_09458_01_020301_20211110T023151.nc',
                   './201908/S5P_PAL__L2__NO2____20190811T022605_20190811T040734_09459_01_020301_20211110T023157.nc'],
                   group='PRODUCT',
                   concat_dim='time',
                   combine='nested',
                   )

image

However, the saved NetCDF file by scn.save_datasets() missed the time dimension for several variables.

To Reproduce

scn = Scene(['./201908/S5P_PAL__L2__NO2____20190811T022605_20190811T040734_09459_01_020301_20211110T023157.nc'], reader='tropomi_l2')

scn.load(['nitrogendioxide_tropospheric_column', 'air_mass_factor_clear', 'cloud_pressure_crb'])

scn.save_datasets(filename='check_time.nc', writer='cf')

Expected behavior

All GEO2D variables have the time dimension.

Actual results

>>> ncdump -h check_time.nc

netcdf check_time {
dimensions:
    time = 1 ;
    y = 4173 ;
    x = 450 ;
    bnds_1d = 2 ;
variables:
    int64 time(time) ;
        time:bounds = "time_bnds" ;
        time:standard_name = "time" ;
        time:units = "days since 2019-08-11" ;
        time:calendar = "proleptic_gregorian" ;
    float longitude(y, x) ;
        longitude:_FillValue = NaNf ;
        longitude:name = "longitude" ;
        longitude:standard_name = "longitude" ;
        longitude:units = "degrees_east" ;
    float latitude(y, x) ;
        latitude:_FillValue = NaNf ;
        latitude:name = "latitude" ;
        latitude:standard_name = "latitude" ;
        latitude:units = "degrees_north" ;
    float air_mass_factor_clear(y, x) ;
        air_mass_factor_clear:_FillValue = NaNf ;
        air_mass_factor_clear:ancillary_variables = "tm5_tropopause_layer_index" ;
        air_mass_factor_clear:end_time = "2019-08-11 04:07:34" ;
        air_mass_factor_clear:file_key = "PRODUCT/SUPPORT_DATA/DETAILED_RESULTS/air_mass_factor_clear" ;
        air_mass_factor_clear:file_type = "tropomi_l2" ;
        air_mass_factor_clear:long_name = "Air mass factor for the cloud-free part of the scene" ;
        air_mass_factor_clear:modifiers = "" ;
        air_mass_factor_clear:platform_shortname = "S5P" ;
        air_mass_factor_clear:reader = "tropomi_l2" ;
        air_mass_factor_clear:sensor = "tropomi" ;
        air_mass_factor_clear:start_time = "2019-08-11 02:26:05" ;
        air_mass_factor_clear:units = "1" ;
        air_mass_factor_clear:coordinates = "latitude longitude" ;
    float cloud_pressure_crb(y, x) ;
        cloud_pressure_crb:_FillValue = NaNf ;
        cloud_pressure_crb:end_time = "2019-08-11 04:07:34" ;
        cloud_pressure_crb:file_key = "PRODUCT/SUPPORT_DATA/INPUT_DATA/cloud_pressure_crb" ;
        cloud_pressure_crb:file_type = "tropomi_l2" ;
        cloud_pressure_crb:long_name = "Cloud optical centroid pressure" ;
        cloud_pressure_crb:modifiers = "" ;
        cloud_pressure_crb:platform_shortname = "S5P" ;
        cloud_pressure_crb:proposed_standard_name = "air_pressure_at_cloud_optical_centroid" ;
        cloud_pressure_crb:reader = "tropomi_l2" ;
        cloud_pressure_crb:sensor = "tropomi" ;
        cloud_pressure_crb:start_time = "2019-08-11 02:26:05" ;
        cloud_pressure_crb:units = "Pa" ;
        cloud_pressure_crb:coordinates = "latitude longitude" ;
    float nitrogendioxide_tropospheric_column(time, y, x) ;
        nitrogendioxide_tropospheric_column:_FillValue = NaNf ;
        nitrogendioxide_tropospheric_column:ancillary_variables = "nitrogendioxide_tropospheric_column_precision air_mass_factor_troposphere air_mass_factor_total averaging_kernel" ;
        nitrogendioxide_tropospheric_column:end_time = "2019-08-11 04:07:34" ;
        nitrogendioxide_tropospheric_column:file_key = "PRODUCT/nitrogendioxide_tropospheric_column" ;
        nitrogendioxide_tropospheric_column:file_type = "tropomi_l2" ;
        nitrogendioxide_tropospheric_column:long_name = "Tropospheric vertical column of nitrogen dioxide" ;
        nitrogendioxide_tropospheric_column:modifiers = "" ;
        nitrogendioxide_tropospheric_column:multiplication_factor_to_convert_to_molecules_percm2 = 6.02214e+19f ;
        nitrogendioxide_tropospheric_column:platform_shortname = "S5P" ;
        nitrogendioxide_tropospheric_column:reader = "tropomi_l2" ;
        nitrogendioxide_tropospheric_column:sensor = "tropomi" ;
        nitrogendioxide_tropospheric_column:standard_name = "troposphere_mole_content_of_nitrogen_dioxide" ;
        nitrogendioxide_tropospheric_column:start_time = "2019-08-11 02:26:05" ;
        nitrogendioxide_tropospheric_column:units = "mol m-2" ;
        nitrogendioxide_tropospheric_column:coordinates = "latitude longitude" ;
    double time_bnds(time, bnds_1d) ;

// global attributes:
        :history = "Created by pytroll/satpy on 2021-12-17 22:22:02.978917" ;
        :Conventions = "CF-1.9" ;
}

Environment Info:

zxdawn commented 2 years ago

This is caused by the how xarray reads variables in group:

xr.open_dataset('./201908/S5P_PAL__L2__NO2____20190811T022605_20190811T040734_09459_01_020301_20211110T023157.nc',
                group='PRODUCT/')['nitrogendioxide_tropospheric_column']

image

xr.open_dataset('./201908/S5P_PAL__L2__NO2____20190811T022605_20190811T040734_09459_01_020301_20211110T023157.nc',
                group='PRODUCT/SUPPORT_DATA/DETAILED_RESULTS/')['air_mass_factor_clear']

image

The coordinates are missing in the subgroup.

zxdawn commented 2 years ago

Submitted the issue on xarray.