search

kevguy / Hangout-Reader

8 stars 1 forks source link

Parsing huge files #2

Open mxxcon opened 7 years ago

mxxcon commented 7 years ago

My hangouts.json is 700MB. When I try to load it using Chrome, the whole tab crashes after 2-3sec. When I try to load it using Firefox it works for 30-40sec and then dev console shows:

allocation size overflow b382d08207bb88ec5603.worker.js:1:494

At which point cpu usage stops and no more progress.

kevguy commented 7 years ago

I can't replicate your problem since I don't have a hangouts.json that big, the hangouts.json I used for development was only about 60mb and the web worker handles it quite well, so a 700mb hangouts.json is much bigger than I expected.

Therefore, I can only guess. Web worker itself has no memory limit, but browsers do, and I think 700mb is definitely bigger that how much they want to handle. How about you try the solution proposed here first: https://kongaraju.blogspot.hk/2012/07/large-file-upload-more-than-1gb-using.html

If the problem is not fixed, I'll try to figure out something else.