search

zgrossbart / jdd

A semantic JSON compare tool
http://www.jsondiff.com
Apache License 2.0
1.05k stars 183 forks source link

Comparing very large JSON documents causes the browser to run out of memory. #40

Closed xhendrikg closed 2 years ago

xhendrikg commented 2 years ago

2x 20MB JSONs Crash Firefox Windows 10 :(

zgrossbart commented 2 years ago

@xhendrikg, 20MB is a very large JSON file. It sounds like the browser is running out of memory. You might try Chrome which is better at memory utilization. You could also allocate more memory to your browser, but fundamentally this is just really big JSON file. Is there any way you can split it up?

xhendrikg commented 2 years ago

@xhendrikg, 20MB is a very large JSON file. It sounds like the browser is running out of memory. You might try Chrome which is better at memory utilization. You could also allocate more memory to your browser, but fundamentally this is just really big JSON file. Is there any way you can split it up?

I reduced the file size to 2mb very easily. I want to compare every hour if the content json file of a game has changed. Is there a way to automate this Process. Also I want to just have one object to be compared and not the whole file. 18 mb where a "random" code I don't need.

zgrossbart commented 2 years ago

There are some NodeJS implementations of this functionality. If you want to do an automated compare you might try that.

There's no way to compare just a part of a file. You would need to edit the file ahead of time to remove the parts you aren't interested in comparing.

xhendrikg commented 2 years ago

There are some NodeJS implementations of this functionality. If you want to do an automated compare you might try that.

There's no way to compare just a part of a file. You would need to edit the file ahead of time to remove the parts you aren't interested in comparing.

Is there something in node js that don't marks every line new when there just got a line added in between?

zgrossbart commented 2 years ago

There is a json-diff package in NodeJS here: https://www.npmjs.com/package/json-diff