OHDSI / GIS

https://ohdsi.github.io/GIS
Apache License 2.0
10 stars 9 forks source link

[Data Source Request]: Unregulated Contaminant Monitoring Rule 3 (2013-2015) #212

Open kzollove opened 1 year ago

kzollove commented 1 year ago

Organization

EPA

Organization Subset

EPA UCMR

Data Source Name

ucmr-3-occurrence-data

Version

No response

Geometry Type

No response

Download Method

direct download

Download file type

txt

Download File Name

UCMR3_All.txt

Download URL

https://www.epa.gov/sites/default/files/2017-02/ucmr-3-occurrence-data.zip

Documentation URL

https://www.epa.gov/sites/default/files/2017-02/documents/ucmr3-data-summary-january-2017.pdf

kzollove commented 1 year ago

This is an interesting case.

In the zip file that is downloaded, there is the main data file (listed above), as well as a file with zipcodes related to site ids UCMR3_ZipCodes.txt

These files will need to be merged. Will attr_spec be up to this challenge? I don't think so...

This either necessitates another *_spec (etl_spec) that handles this specific situation, or geom_spec should be repurposed as data_spec (the spec that handles data_sources). Within a data_spec there could be {geom: "..."} that functions the same way as the geom_spec

tibbben commented 1 year ago

yep, the SVI data is similar in some ways (one file name for zip file, another for shape file inside the zip. Also the data I am using for local temperature and NDVI in Miami-Dade has several files(data layers) in one zip file. I think we can handle this, but it will require a few more custom fields/etl functions. With that said, if we think carefully, we should be abole to handle any number of similar cases ... we can chat if you want.