ua-snap / rasdaman-ingest

Collection of ingredients/configurations + docs for ingesting data into Rasdaman
MIT License
4 stars 0 forks source link

Arctic eds mean annual temp #27

Closed kyleredilla closed 2 years ago

kyleredilla commented 2 years ago

This PR contains a preprocessing notebook and ingest recipe for a new mean_annual_temp coverage to supply the data for the forthcoming "Mean Annual Temperature" plate of the Arctic EDS.

This coverage contains historical and projected mean annual temperature data derived from SNAP downscaled CRU-TS40 and CMIP5 datasets. No styles are tracked here at this time. This coverage has not yet been implemented on Zeus. Copies of the necessary ingest data exist on Atlas at /atlas_scratch/cparr4/rasdaman_prod

The recipes in this PR are good examples for ingesting big GeoTIFF stacks that use a "real" time axis.

kyleredilla commented 2 years ago

Hey @cstephen could you post a screenshot of the "maps" that worked before you tried executing the histograms cell?

cstephen commented 2 years ago

Here are the map plots that work, before the histogram step that fails:

image

kyleredilla commented 2 years ago

image

@cstephen I have not been able to reproduce the data produced from your execution of the notebook. It looks like it is creating -inf values where the nodata values should be.

Also, the resulting summarized arrays are different:

image

I made a small change, I don't think it will make a difference, but I wonder if you could try running the notebook again when you get the chance? I'm not sure what else to try on my end.

cstephen commented 2 years ago

I pulled down the latest changes to this branch and I'm getting a bunch of these warnings now:

RuntimeWarning: overflow encountered in reduce

image

I'm guessing these warnings are related to the differences in my output files. I'll see if I can fix this on my end since it sounds like something is wrong with my environment. Like before, the maps are displayed properly, but the histograms are showing the same error.

The unique file sizes are different this time, too, but still do not match the expected file sizes:

image