Soil survey -- covariate calculation

[sigmafelix]

• Target: Gridded National Soil Survey Geographic Database (gNATSGO)
• File footprint: 8.5 GB compressed (75 GB uncompressed)
• Computation strategy: using file proxy instead of reading the entire raster into memory

[sigmafelix]

The File Geodatabase in DDN includes raster layers, which could be only read by ArcGIS* products. Soil survey data is stored in multiple CSV files and a reference raster with unique identifiers that relate CSV attributes. One alternative is using Microsoft Planetary Computer, which I am not very familiar with. Will look at file-based options along with leveraging Planetary Computer.

[sigmafelix]

For AZO covariate calculation, the calculation strategy is to extract unique identifiers (map unit key or mukey) first, then join attributes from tables with mukeys. Alternative non-memory-intensive methods are required, as some tables are too large to transmit and process. One quick idea is to make a file-based database in the scratch storage at the geo cluster and then query it. I am also listing up tables for the join operation out of 70 attribute tables in gNATSGO. It looks like some tables are full of descriptions, which we probably don't want to use.

Soil properties and pesticides cf. https://doi.org/10.1016/j.ceja.2022.100301