search

mapbox / geojsonhint

IMPORTANT: This repo will be archived. Use @placemarkio/check-geojson instead
ISC License
258 stars 37 forks source link

Big file support for /geojsonhint #13

Closed tmcw closed 8 years ago

tmcw commented 11 years ago

geojsonhint is fast enough to process big files but the DOM/CodeMirror is bringing me down. I should look into what it'd mean to have just errors / success shown and small chunks of context instead of the full file.

vsivsi commented 10 years ago

This would be great. The osmtogeojson project had a similar issue a while back when processing JSON inputs larger than ~256MB. There are internal limits on the length of a string in node.js (1GB if memory serves) that can be overcome during serialization/deserialization by using a streaming parser like JSONStream. See this issue for the full discussion and solution they used: https://github.com/tyrasd/osmtogeojson/issues/17

tmcw commented 8 years ago

Closing - I tried out JSON streaming parsers and while they're great, I don't think they hit a very good performance / stability point. A big-file compatible version of geojsonhint will likely be in a different language.