paulmillr / readdirp

Recursive version of fs.readdir with streaming api.
https://paulmillr.com
MIT License
378 stars 51 forks source link

Feature: Refactor this to use fs.openDir in future and not fs.readdir #105

Closed frank-dspeed closed 3 years ago

frank-dspeed commented 4 years ago

NodeJS > 12.11.1 will have async readdir support Landed in cbd8d71 ( nodejs/node@cbd8d71 ) as fs{Promises}.opendir(), which returns an fs.Dir, which exposes an async iterator.

to stay compatible we could implament it with the same stream interface and simply check if fs.openDir is there if not fallback to existing implamentation.

This is a lobby Request <3 by the microsoft/vscode community

paulmillr commented 4 years ago

easy

paulmillr commented 4 years ago

See branch v4 for implementation: https://github.com/paulmillr/readdirp/tree/v4

Execute node examples/list, ensure to pass big directory there.

  1. v4 branch ditches readable stream in favor of async generators (node 10+).
  2. This allows to speed-up a file tree of 45k entries from 1.6 sec to 500ms (3x).
  3. opendir works, you'll need node 12.12, it's 3x slower than readdir — MEH! It's as slow as current readdirp-v3.
yisibl commented 1 year ago

Hi, @paulmillr Why is opendir slower instead?

paulmillr commented 1 year ago

@yisibl ?

yisibl commented 1 year ago

opendir works, you'll need node 12.12, it's 3x slower than readdir — MEH! It's as slow as current readdirp-v3.

paulmillr commented 1 year ago

What is the question? Open dir is 3x slower. Why is it slower? I don't know. Shitty node.js implementation.