On really big files (60MB) the browser stalls

htm-community / nupic.visualizations

Web application for interactive graphs, anomaly highlighting and online monitoring.

MIT License

17 stars 11 forks source link

On really big files (60MB) the browser stalls #56

Closed breznak closed 8 years ago

breznak commented 8 years ago

As @jefffohl mentioned in #27 , the functionality for parsing large files is much improved by that PR, but for a really big file (60MB+) the browser gets on the knees.

To improve this, we could

use subsampling, similar to #43
have a MAX_SIZE field and if the graph is larger than that
- switch to the "sliding window" mode

jefffohl commented 8 years ago

I am not sure which option is the best. I am leaning towards the windowing option. We can post an alert stating that the data is being windowed, and allow the user to turn that off, and load the entire file if they really want to try that - knowing that it might crash the browser.

What option do you think users would prefer?

breznak commented 8 years ago

@jefffohl I don't know either. The widowing option will be used anyway for the streaming data, so we can start with that and enable #43 later?

jefffohl commented 8 years ago

@breznak - we can automatically use windowing on files over a certain size. I am going to arbitrarily set this to 5MB.

breznak commented 8 years ago

Fixed with #64 , large files use 5MB size window