rocket-connect / for-emit-of

Turn Node.js Events into Async Iterables.
https://www.npmjs.com/package/for-emit-of
MIT License
9 stars 2 forks source link

Performance improvement over fast iterables #24

Closed Farenheith closed 2 years ago

Farenheith commented 2 years ago

Solves #23

Some performance improvements were implemented in this PR.

I also added a new test suite to validate performance, using a large file as the source of event emitting. This suite doesn't pass on the previous version, but I don't know how long it would take to finish because it timed out in 100 seconds. I think it would take 5 to 10 minutes to finish. Using this branch locally the test finished in 386 ms with noSleep = false and 349ms with noSleep = true.

Running a separated benchmark without all the mocha overload I got the following results, but over a file with only 9074 lines:

before

####################################################
WARMUP: Current memory usage: 2.2 MB
WARMUP: readline forEmitOf profiling started
Read 9074 lines
WARMUP: readline forEmitOf profiling finished in 10757ms
WARMUP: Current memory usage (before GC): 15.15 MB
WARMUP: Current memory usage (after GC): 2.39 MB

####################################################
RUN 1: readline forEmitOf profiling started
Read 9074 lines
RUN 1: readline forEmitOf profiling finished in 10805ms
RUN 1: Current memory usage (before GC): 11.14 MB
RUN 1: Current memory usage (after GC): 2.39 MB

####################################################
RUN 2: readline forEmitOf profiling started
Read 9074 lines
RUN 2: readline forEmitOf profiling finished in 10798ms
RUN 2: Current memory usage (before GC): 11.91 MB
RUN 2: Current memory usage (after GC): 2.4 MB

####################################################
RUN 3: readline forEmitOf profiling started
Read 9074 lines
RUN 3: readline forEmitOf profiling finished in 10849ms
RUN 3: Current memory usage (before GC): 14.5 MB
RUN 3: Current memory usage (after GC): 2.66 MB

after

####################################################
WARMUP: Current memory usage: 2.21 MB
WARMUP: readline forEmitOf profiling started
Read 9074 lines
WARMUP: readline forEmitOf profiling finished in 56ms
WARMUP: Current memory usage (before GC): 4.21 MB
WARMUP: Current memory usage (after GC): 2.35 MB

####################################################
RUN 1: readline forEmitOf profiling started
Read 9074 lines
RUN 1: readline forEmitOf profiling finished in 47ms
RUN 1: Current memory usage (before GC): 6.75 MB
RUN 1: Current memory usage (after GC): 2.62 MB

####################################################
RUN 2: readline forEmitOf profiling started
Read 9074 lines
RUN 2: readline forEmitOf profiling finished in 52ms
RUN 2: Current memory usage (before GC): 6.32 MB
RUN 2: Current memory usage (after GC): 2.63 MB

####################################################
RUN 3: readline forEmitOf profiling started
Read 9074 lines
RUN 3: readline forEmitOf profiling finished in 46ms
RUN 3: Current memory usage (before GC): 4.8 MB
RUN 3: Current memory usage (after GC): 2.64 MB

after with noSleep

####################################################
WARMUP: Current memory usage: 2.21 MB
WARMUP: readline forEmitOf profiling started
Read 9074 lines
WARMUP: readline forEmitOf profiling finished in 23ms
WARMUP: Current memory usage (before GC): 3.39 MB
WARMUP: Current memory usage (after GC): 2.33 MB

####################################################
RUN 1: readline forEmitOf profiling started
Read 9074 lines
RUN 1: readline forEmitOf profiling finished in 16ms
RUN 1: Current memory usage (before GC): 4.38 MB
RUN 1: Current memory usage (after GC): 2.35 MB

####################################################
RUN 2: readline forEmitOf profiling started
Read 9074 lines
RUN 2: readline forEmitOf profiling finished in 14ms
RUN 2: Current memory usage (before GC): 4.18 MB
RUN 2: Current memory usage (after GC): 2.38 MB

####################################################
RUN 3: readline forEmitOf profiling started
Read 9074 lines
RUN 3: readline forEmitOf profiling finished in 16ms
RUN 3: Current memory usage (before GC): 4.21 MB
RUN 3: Current memory usage (after GC): 2.37 MB
danstarns commented 2 years ago

Thank you for this! Sorry, I have been distant on your raised issue #23. I will review it soon! Cheers @Farenheith 🍻

Farenheith commented 2 years ago

@danstarns no problems, mate! I was planning to work on it, I just opened the issue to don't forget about it!

Farenheith commented 2 years ago

Nice! Thank you, and happy new year!

danstarns commented 2 years ago

Nice! Thank you, and happy new year!

Happy new year to you too.