okdistribute / render-data

Intelligently render a data stream in a browser.
15 stars 6 forks source link

[SCALING] todo: researching/profiling .pause() method used in csv stream render #3

Open laurengarcia opened 8 years ago

laurengarcia commented 8 years ago

NOTE: this is just a reminder to look into this later; stream .pause() implementation should be fine for demos, and someone else on team might say "don't worry, garbage collection has your back".

this is more a question for scaling, use case would be user opens a bunch of tabs all at once and previews a large csv in each. the stream goes into memory buffer, gets paused via stream.pause() and then sits there as the user opens 10 more tabs, same thing. at what point does browser start crawling?

so the csv render is an adaption of this piece of code from mafintosh: https://github.com/mafintosh/hyperdrive/blob/gh-pages/index.js#L142

it uses the .pause() method on the stream and leaves open the possibility that the user might never unpause it.

my instinct is to create a timeout that destroys the stream after x number of milliseconds and then if the user wants to interact with the data again, i would re-init the stream. maybe garbage collection has my back. wondering what mafintosh will say.

node docs: https://nodejs.org/api/stream.html#stream_readable_pause

okdistribute commented 8 years ago

good point!