search

ua-snap / rasdaman-ingest

Collection of ingredients/configurations + docs for ingesting data into Rasdaman
MIT License
4 stars 0 forks source link

Ingest for IEM/tas_pr_2km maybe has seasonal luts flipped? #8

Closed brucecrevensten closed 2 years ago

brucecrevensten commented 2 years ago

https://github.com/ua-snap/rasdaman-ingest/blob/cb6ed1ae71c13b2a9be18e6b85ea6521f08783a2/iem/tas_pr_2km/luts.py#L40

In that code, the seasons don't seem to follow what I'd pick for the logical order, i.e. should it be MAM=1, JJA=2? Then the seasons go 0, 1, 2, 3 for winter, spring, summer, fall.

brucecrevensten commented 2 years ago

This is also true in the historical ingest.

kyleredilla commented 2 years ago

So this was done to match the order in which files are ingested into rasdaman. This is one downside to GeoTIFF ingestion, and I made a note of it in the "GeoTIFF Stack" section of the Rasdaman usage doc. Rasdaman imports the files in a sorted order but you cannot insert on any axis, only prepend or append. So if e.g. DJF = 0 , MAM = 1, and JJA = 2, which would be intuitive, rasdaman would construct the coverage reading DJF files first then JJA files (because those come next based on sorting of filenames) and then halt with error when it got to the MAM files because it can't insert between 0 and 2. I didn't dig that deeply into this so there might be a workaround, but I'll close for now with plans to implement #11 which will help reduce confusion around this quirk. This is currently handled in the API by relying on hard-coded encodings matching those in the luts.py of the relevant ingest files here.