Increase efficiency of HMS Grid Conversion script

[nick-j-roberts]

Current implementation takes a long time to process records, requires conversion of zarr data to DSS for local download

• Should be using precipitation DSS files created during SST processing rather than regenerating
  • Difficulty here is including temperature records in these DSS file
  • Potential solve: just write separate DSS files out for temperature and precip data and reference both with GRID file
• Should also try to take advantage of preexisting POR DSS records on s3 when available
  • Would simplify the process to pulling precip DSS from s3 included for SST processing and pulling DSS for periods identified by query from POR for temperature, then merging these together into a single DSS file, or DSS files separated by temperature and precip

[nick-j-roberts]

Minor efficiency improvement: reduce the frequency of loading aoi geometry from watershed to use in trimming datasets, just do it once

[nick-j-roberts]

Idea:

• query meilisearch for start and end dates of storm periods
• iterating through date windows:
  • async send request to process api with AORC temp extraction for periods of interest
  • while awaiting request response, grab precip dss from s3 associated with SST run
  • when temperature response is ready, combine precip and temp dss files into single file
  • store in dss folder referenced by grid file
  • register dss file and pathnames within in grid file

using process api and asynchronous processing could potentially improve performance, needs testing -- spin up of docker associated with process api could slow this down

[nick-j-roberts]

Change in approach to utilize plugin workflow and take advantage of batch processing makes this issue not relevant to updated approach