Degree Days Update

[charparr]

This PR of somewhat misleadingly named branch (these are not actually Prefect'd) represents a replacement of all prior degree day coverages. Ingest recipes include style hooks for all current EDS and ARDAC styles.

The coverages that will be deprecated are as follows:

• heating_degree_days
• degree_days_below_zero
• freezing_index
• thawing_index

These coverages are their replacements:

• heating_degree_days_Fdays
• degree_days_below_zero_Fdays
• air_freezing_index_Fdays
• air_thawing_index_Fdays

• Each coverage has its own ingest recipe and source data, though they share a common luts.py. To test this PR execute you will need to run four total ingests, one per coverage.

Testing Notes

• I've been running these ingests on Zeus as my own user cparr4@datacubes out of this directory /home/UA/cparr4/cparr4-tmp/zipped

• If you are also testing as your own user, run this command before ingesting to avoid the style hooks permission-failing" echo "-u "$RASADMIN > $HOME/.curlrc

• Sort out your conda environment as needed:

  • eval "$(/opt/miniconda3/bin/conda shell.bash hook)"
  • conda activate rasdaman
  • Because we have GeoTIFF merging script in the loop, your env will need the following packages:
    • pandas
    • rasterio
    • xarray
    • tqdm
    • netcdf4 so just conda install these if you don't have them present already.

• Grab the source data from Shared/Tech_Projects like so:

  rsync -av --usermap=*:$(id -u -n) /workspace/Shared/Tech_Projects/Degree_Days_NCAR12km/heating_degree_days.zip .
  rsync -av --usermap=*:$(id -u -n) /workspace/Shared/Tech_Projects/Degree_Days_NCAR12km/degree_days_below_zero.zip  .
  rsync -av --usermap=*:$(id -u -n) /workspace/Shared/Tech_Projects/Degree_Days_NCAR12km/air_freezing_index.zip  .
  rsync -av --usermap=*:$(id -u -n) /workspace/Shared/Tech_Projects/Degree_Days_NCAR12km/air_thawing_index.zip .

  • Next, unzip the compressed data. Don't do this for all the data at once, or make a separate ingest directory for each coverage. I'll use heating degree days as my example from here on out: unzip heating_degree_days.zip
  • Merge the GeoTIFFs into a NetCDF to facilitate a faster and more gentle ingest process: python ~/rasdaman-ingest/arctic_eds/degree_days/heating_degree_days_Fdays/merge.py
  • export LUTS_PATH=~/rasdaman-ingest/arctic_eds/degree_days/heating_degree_days_Fdays/luts.py
  • Run the ingest; in my experience these each only take a few seconds to complete: wcst_import.sh -i /opt/rasdaman/etc/.rasadmin ingest.json