paulmillr / readdirp

Recursive version of fs.readdir with streaming api.
https://paulmillr.com
MIT License
378 stars 51 forks source link

How to write emitted output data into the Node's pipeline correctly? #172

Closed aleksey-hoffman closed 3 years ago

aleksey-hoffman commented 3 years ago

Hey @paulmillr Sorry for asking questions here, but I'm having troubles returning data from the emitter.

I'm trying to recursively walk a directory and write all paths into a file using this module and Node's pipeline.

const readdirp = require('readdirp')
const zlib = require('zlib')
const { pipeline } = require('stream')

let w_stream = fs.createWriteStream('C:/test/paths.txt')
let r_stream = readdirp('C:/Program Files')

pipeline(
  r_stream,
  zlib.createGzip(),
  w_stream,
  (error) => {
    if (error) {
      console.error('Pipeline failed', error)
    } 
    else {
      console.log('Pipeline succeeded')
    }
  }
)

The r_stream outputs objects like {fullPath: String, stats: Object} but the pipe expects strings. So, I get the following error:

The "chunk" argument must be of type string or an instance of Buffer or Uint8Array. Received an instance of Object

Question

How do I process r_stream data chunks so it returns entry.fullPath (strings) instead of entry (object) into the pipeline? I tried adding the following, but I get the same error:

...
r_stream.on('data', (entry) => {
  w_stream.write(`${entry.fullPath}\n`)
})

pipeline(
  ...

How do I properly return the entry.fullPath string from r_stream.on('data'... into the pipeline?

paulmillr commented 3 years ago

See this: https://nodejs.org/api/stream.html#stream_object_mode