openaddresses / batch

OpenAddresses/Machine based AWS Batch based ETL Processing
https://batch.openaddresses.io/
MIT License
6 stars 5 forks source link

Invalid GeoJSON download #208

Closed stefanb closed 3 years ago

stefanb commented 3 years ago

Describe the bug Downloaded GeoJSON is malformed. Example (shortened to first 5 lines):

{"type":"Feature","properties":{"hash":"3d653f29546cba89","number":"1","street":"Cankarjev trg","unit":"","city":"Ajdovščina","district":"Ajdovščina","region":"Goriška","postcode":"5270","id":"11028489"},"geometry":{"type":"Point","coordinates":[45.8873882,13.9095556]}}
{"type":"Feature","properties":{"hash":"84d57df68788ae72","number":"2","street":"Cankarjev trg","unit":"","city":"Ajdovščina","district":"Ajdovščina","region":"Goriška","postcode":"5270","id":"11028497"},"geometry":{"type":"Point","coordinates":[45.8875309,13.909424]}}
{"type":"Feature","properties":{"hash":"705e85434dd18813","number":"2A","street":"Cankarjev trg","unit":"","city":"Ajdovščina","district":"Ajdovščina","region":"Goriška","postcode":"5270","id":"19574024"},"geometry":{"type":"Point","coordinates":[45.8874504,13.9094771]}}
{"type":"Feature","properties":{"hash":"8d092860be43a431","number":"3","street":"Cankarjev trg","unit":"","city":"Ajdovščina","district":"Ajdovščina","region":"Goriška","postcode":"5270","id":"11028519"},"geometry":{"type":"Point","coordinates":[45.8875676,13.9095006]}}
{"type":"Feature","properties":{"hash":"941576f3a28d17a1","number":"4","street":"Cankarjev trg","unit":"","city":"Ajdovščina","district":"Ajdovščina","region":"Goriška","postcode":"5270","id":"11028543"},"geometry":{"type":"Point","coordinates":[45.8876134,13.909577]}}
...
  1. It has no basic structure:
    {
    "type": "FeatureCollection",
    "features": [
    ...
    ]
    }
  2. It is missing commas between features.

To Reproduce Steps to reproduce the behaviour:

  1. Go to job preview page (eg https://batch.openaddresses.io/job/161039 )
  2. Click on download icon
  3. Scroll down to GeoJSON option and click it
  4. Inspect the downloaded *.geojson.gz file

Expected behavior Expected valid GeoJSON

Screenshots image

Desktop (please complete the following information):

Additional context Note that coordinates in the above example are flipped because of #90 & https://github.com/openaddresses/openaddresses/issues/5917

iandees commented 3 years ago

The "GeoJSON" download is newline-delimited GeoJSON, which allows for more efficient reading of large collections of GeoJSON.

We probably could do a better job documenting this, but any GIS software written in the last 5 years or so should be able to read it without a problem.

stefanb commented 3 years ago

If this is the case it should be stated clearly somewhere near the download or in filename extension.