USDA NASS - Covariate Calculation

[sigmafelix]

Results

• 15 tiff files (2008-2022)
• HUCS: 8, 10, and 12
• Covariates were calculated in HPC. Runtime was 6:53:37 and peak memory ~400GB.
  • Configuration: Apptainer sif image with ddn mounted, 15 cores, 450GB assigned
• Data is in my ddn space as it seems I have no writing privilege for the group directory.

TODO

• Clean column names

[kyle-messier]

@sigmafelix Our group has read, write, execute access to /ddn/gs1/group/set/Projects/PrestoGP_Pesticides Please check that again and if you still don't have write access, I'll reach out to OSC.

[sigmafelix]

@Spatiotemporal-Exposures-and-Toxicology Permission settings seem to be differently applied to each subdirectory...

[kyle-messier]

@sigmafelix ok, I thought recursive permissions within subfolders was the default. I made a change to hopefully apply the permission through the entire project directory. Please try again and let me know. Thanks!

[kyle-messier]

@sigmafelix Can you provide the NASS code to the group ddn directory for PrestoGP or share via email?

[sigmafelix]

@kyle-messier https://github.com/NIEHS/targets_PrestoGP/blob/main/code/02_Geographic_Covariates/Calc_HUC_extract_test.r

It seems not to be renamed before. The workflow of that code is--

1. For i $\in$ 1:|HUC04| do
2.   Split lower-level HUC areas by i
3.     Using lower-level HUCs in i, read HUC areas
4.     Calculate zonal frequencies of each crop class by `terra::freq()`
5.     Clean the extracted data
6.     Bind rows

1-6 is parallelized for each year's NASS tiff file.

[kyle-messier]

@sigmafelix Thanks. I'll take a look. Converting that to a basic targets workflow may be a lift.

[sigmafelix]

@kyle-messier I could add it to the pipeline in main. Please let me know after you will push all the changes made in the meantime.