healthyregions / oeps

Opioid Environment Policy Scan - data explorer and backend management
https://oeps.healthyregions.org
GNU General Public License v3.0
0 stars 0 forks source link

geodata pipeline #34 #38

Open bucketteOfIvy opened 5 days ago

bucketteOfIvy commented 5 days ago

Major changes in brief:

Taken altogether this should resolve my portion of #34, with the get_geodata command now returning the correct file types. The only two things I cannot currently confirm function are the zcta geodata download -- as one of the needed files is ~500MB in size and takes forever to download -- and the pmtiles export -- as I do not have tippecanoe on my system. The latter isn't an issue for #34, but the former I'll try to resolve by just leaving this running overnight to see what the current output format is.

netlify[bot] commented 5 days ago

Deploy Preview for oeps canceled.

Name Link
Latest commit c06cc5c3384fc1770760978d7cc57cb9eaf50ec6
Latest deploy log https://app.netlify.com/sites/oeps/deploys/668586f021c34200080cc3e8
bucketteOfIvy commented 4 days ago

Oookay, tested this on the zcta data, and it does not complete in a time span of less than 6 hours.* Whatever is breaking seems to break during the file transfer process, so that I'm guessing something weird is happening on the Census' end of things (seeing as it only fails on one file from specifically the ZCTA data).

*I can't confirm that it completes after 6 six hours, just that it doesn't complete by the sixth hour.

mradamcox commented 4 days ago

Yeah, no worries, it definitely shouldn't take anywhere close to that long so something is up. I'll look into it when I get a chance to check out this PR.