search

openaddresses / batch

OpenAddresses/Machine based AWS Batch based ETL Processing
https://batch.openaddresses.io/
MIT License
6 stars 5 forks source link

s3 url for validated downloads #250

Open missinglink opened 2 years ago

missinglink commented 2 years ago

are there s3 versions of the validated data downloads? I tried taking an s3 url and changing the filename from source.geojson.gz to validated.geojson.gz without success.

ingalls commented 2 years ago

Hey @missinglink, sorry busy work week. I'm hoping to get to S3 validation this coming weekend. At the moment sources that are processed successfully will generate both a normal source.geojson.gz file and a validated address file and store it, but only the source file is currently accessible.

ingalls commented 2 years ago

@missinglink Validatd data downloads should now work. Please note that this feature is not retroactive. Sources that haven't run successfully since the validated processor was introduced will not currently return a successful object.

I've also updated openaddresses/lib with the new endpoints.

OA_TOKEN='<redacted>' ./oa job validated --:job 123 --script > out.geojson.gz
ingalls commented 2 years ago

I'll update the API with an S3 location shortly.