search

thecodrr / fdir

⚡ The fastest directory crawler & globbing library for NodeJS. Crawls 1m files in < 1s
https://thecodrr.github.io/fdir/
MIT License
1.51k stars 58 forks source link

Make a note about the `sync` api being much slower #68

Closed vjpr closed 2 years ago

vjpr commented 3 years ago

I think a lot of my early testing of this package was using sync api which made me think it was not that fast.

It's common for a Node dev to reach for sync apis for file access in cli tools...as you wouldn't normally think async would give speed gains, unless you are running in a webserver where you don't want to block the event loop.

But for fdir it offers huge benefits in speed when the threadpool is increased. On macOS at least.

UV_THREADPOOL_SIZE=8

776457
fdir: 2.336s
776457
fdir.sync: 8.799s

UV_THREADPOOL_SIZE=2

776457
fdir: 6.870s
776457
fdir.sync: 7.804s

Crazy its set so low. https://www.sebastienvercammen.be/your-libuv-thread-pool-size-is-too-small/

Also, make a note of increasing UV_THREADPOOL_SIZE.