search

pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.06k stars 292 forks source link

Calculate solar angles if requested on load() #77

Open pnuu opened 7 years ago

pnuu commented 7 years ago

Some of the readers can already return Solar zenith/azimuth (and satellite zenith) angles if requested in the load() call. In these cases the data are readily available in the files, but it would be awesome to have them available for all the readers. At least the Solar zenith angles can be calculated using pyorbital, and are already used in many composites for adjusting illumination. The illumination correction is done using modifiers for each channel, and I can't see if that mechanism can be used to blend two channels together.

An alternate solution I see for the DayNightCompositor is to make the solar_zenith_angle an optional prerequisite and calculate the angles if they are not available. The upside off this method is that the memory footprint is somewhat lower, as the angles are present only when the composite is generated, but then again the angles would need to be cached for other composites using DayNightCompositor.

In any case, the load()-able procedural/virtual/implicit/indirect/deducible/inferred// datasets would be quite a useful feature.

Any thoughts?

djhoese commented 7 years ago

What if we had a utility function/class that created and cached SZA when provided a Dataset object? It could use the metadata from the dataset to figure everything out. Not sure if this would provide a lot of utility over just calling the pyorbital functions. By having the utility someone who is developing something wouldn't need to set up a compositor/modifier just to get the SZA values. The SunZ corrector modifier stuff could also just use this utility function. Regardless of how it's implemented, we'd have to keep in mind that someone might want to clear whatever cache we set up.

Are you suggesting though that there be an "automated" way of satpy saying "if no solar zenith angle is available, compute it" and the compositor wouldn't have to know where it came from?

pnuu commented 7 years ago

Yes, exactly. There are other similar data that could be calculated based on available information, e.g. satellite alt/az, solar/lunar alt/az, excact timing for each scanline, and maybe some others. At least some of the angles are already used as optional prerequisites in the existing compositors/modifiers, but are only useful for the instruments where the data are readily available. In some cases there might be a reason to have a mechanism to stop calculating those extra datasets, e.g. for saving CPU time and/or memory.

mraspaud commented 7 years ago

I think procedural channels are a good idea. In order to save memory and CPU, I would suggest we develop this on top of PR #78, since procedural arrays are supported in dask. This means that we would describe how to generate the data, but it would actually be computed before it's needed.

mraspaud commented 1 year ago

we had a chat about this again yesterday during the Pytroll community meeting, and noted that there might be ambiguity regarding resolution. So I'm wondering if a solution would be to require the passing of the resolution parameter in that case?

djhoese commented 1 year ago

Pass resolution during the Scene.load call or in a composite definition as a prerequisite? I could see a helper function that takes a single resolution, returns the first available DataID with that resolution and then loads it to be used for generating the angles.

I've seen it referred to in other code as something like this being a "donor". That is, we are loading a DataArray that we aren't going to use but it is donating its geolocation and other metadata to generate angles. I think we also talked in the past about whether or not this generation should happen in the reader or happen as a composite.