search

visionmedia / node-progress

Flexible ascii progress bar for nodejs
MIT License
2.98k stars 221 forks source link

Heavy performance impact? #124

Open slavafomin opened 8 years ago

slavafomin commented 8 years ago

Hello!

Thank you for this great library!

I'm using it to display progress of CSV-file parsing. I have a CSV-file of 800K records and I'm calling tick() for each record. When I'm not using progress bar module it takes ~20 seconds to process the file, however, when I use the progress bar, it takes ~30 seconds! I've tried to increase the renderThrottle to 500 ms, but it's not helping.

thebigredgeek commented 8 years ago

Hmm yeah log output is never free, especially when you are dealing with tokenization etc which this lib does. One suggestion might be to run the progress lib on the "main" process and fork a second progress for the actual file parsing. The second process can push stats back to the main process for output.

nullivex commented 8 years ago

@slavafomin I would note that when I process huge stacks of data with this tool I always tick in blocks.

So I make a counter and only update every thousand such as

if(counter % 1000 === 0) progress.tick(1000)
counter++
piranna commented 8 years ago

What about acumulate changes and show them at a maximum of 60fps as how requestAnimationFrame() polyfills does? In fact, one of them could be used...

andrscrrn commented 8 years ago

The performance impact is HUGE. I just dropped this library because of this. My program was dying because of it. 🙁

thebigredgeek commented 7 years ago

Yeah most people use progress for simple command line tools, not heavy-weight parsing. stdout can be expensive