pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.05k stars 289 forks source link

Improve handling of multiple time steps being passed to the Scene #1668

Open djhoese opened 3 years ago

djhoese commented 3 years ago

Feature Request

Is your feature request related to a problem? Please describe.

The Scene object can only handle one time step of data for sensors like the ones most commonly found on geostationary satellites. This is due to the responsibilities of the Scene and the complications that would arise if the Scene handled more than one time step (that's why the MultiScene exists). However, the Scene doesn't actually do anything to prevent a user from providing more than one time step's worth of files. This results in users getting odd unhelpful errors or very odd and unexpected results. Satpy should be able to inform users of what's going on much earlier.

See #1667 and #1664 and probably many other issues for related discussions.

Describe the solution you'd like An exception should be raised when this is detected. This will most likely require something to be specified in each reader that has this "limitation". There are a couple types of readers that I can think of that exist in Satpy:

  1. Geostationary satellite readers like those for ABI where each "file set" results in file handlers that produce a single whole image with a single AreaDefinition.
  2. Separate scan geostationary readers like SEVIRI (I think) where each "file set" results in multiple file handlers where each file is only producing part (a scan) of the overall image. This is harder to handle as one file handler may not know how many of its "sibling" handlers exist.
  3. Swath based handlers like those for VIIRS or MODIS where the number of granules/files can vary and the resulting image has more or less data depending on how many granules are provided. Technically someone should be able to provide as many granules as they want and they'll just get a really long concatenated array of data. Whether or not the final image looks good after resampling is another issue, but I'm not sure this can or needs to be handled the same as what's discussed here.
  4. Swath based handlers that don't expect for concatenation...do these exist?

Describe any changes to existing user workflow Only errors when there should have been errors already.

Additional context The only other solution to this would be to allow multiple time steps and produce something other than a regular Scene or have the Scene handle it. Maybe returning an xarray Dataset with a time dimension. This gets pretty hand-wavy and magical when done automatically. It should probably be specifically requested by the user.

mraspaud commented 3 years ago

Some thoughts:

djhoese commented 3 years ago

For the SAR grd data, programmatically is that any different than geostationary satellites like ABI?

Another case I don't want to preclude for the future is tiled inputs where concatenation may not be time based or in a single dimension.

mraspaud commented 3 years ago

Yes SAR is different in this case, and joins in in your second comment here, as they could be viewed as tiles. Although sentinel 2 level 1c would be even more representative, as they are tiled by definition.