Open anphanngocvn opened 5 years ago
Hi, I ran into the same problem a few weeks ago. The issue is that you can only add so many arguments to a command (maybe 1000 on bash/linux? I can't remember,). The only solution to this is to do a batch operation (for example, if you have 10,000 files to merge, merge 500 at a time until you have 20 files, and then merge those 20 together). The solution I took was to write a quick node script that iterated over every file in a folder and merges that way. There is not really a max file type restriction in node when iterating over a folder (although you could risk running out of ram!).
I have the same issue on the entire Kantō region (roads only) of Japan. Its just merging two files. not hundreds. Apparently the file itself is too big at 769.3mb
@anphanngocvn @qrcf which version of node are you using? i encountered the problem in v6.10, tried in v11 and it's all working fine. I still trying to find out now which node version was it really fixed.
Might want to remove the 2 from https://github.com/mapbox/geojson-merge/blob/fccdc8d5a53f13f535ee6c538a6edf2d98c57e76/geojson-merge#L19, indentation seems to inflate file sizes.
Might want to remove the 2 from
, indentation seems to inflate file sizes.
Change 2 to 0 works for me! thank you @cagriozkurt
I have a few .json files (some of them about 750Mb) need to be merged into one .geofile. However when I run this command to merge them.
geojson-merge folder/*.geojson > combined.geojson
. I have following errors:`/geojson-merge/geojson-merge:17 process.stdout.write(JSON.stringify(merge(argv._.map(function(n) { ^
RangeError: Invalid string length at JSON.stringify ()
at Object. (/geojson-merge/geojson-merge:17:29)
at Module._compile (internal/modules/cjs/loader.js:701:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)
at Module.load (internal/modules/cjs/loader.js:600:32)
at tryModuleLoad (internal/modules/cjs/loader.js:539:12)
at Function.Module._load (internal/modules/cjs/loader.js:531:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:754:12)
at startup (internal/bootstrap/node.js:283:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)`
How do you tackle this?