Closed JoeGermuska closed 5 years ago
I recommend using newline-delimited JSON input so that you can stream one feature at a time. That will work for larger inputs, though of course you can still run out of memory. You can also pass arguments to Node to give it more memory.
I'm trying to produce a topojson file of all ZCTAs in the US. I converted the Census TIGER data to geojson (1.3 GB), and then when I try to convert it to topojson, I get the error below.
As my next step, I extracted roughly half the data (based on ZCTA ids beginning with 0-4). This produced a 609M geojson file. When executing
geo2topo
on that file, I got a different error:FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
from the v8 JsonParserI tried again, splitting into 5 groups, and got the same error on a geojson file of around 300MB.
When I split into 10 geojson files, they all make it through correctly, but now I realize that I don't know a simple way to put them all back together again!
Can anything be done to produce topojson from large geodata files?