How to properly read chunks of all files content into a single stream?

[Ajaxy]

I'd love to not only get chunks with filenames, but also read all files content by chunks (i.e. with fs.createReadStream) and send chunks subsequently into a single common output stream. Wondering what would be the right and shortest way to do so? Thanks.

Now I'm doing it like this:

const outputStream = through.obj();
const filenamesStream = gs(`${this.dirName}/*`);

filenamesStream.on('end', () => { outputStream.end(); })
filenamesStream.pipe(through2.obj(({ path }, _, cb) => {
    const fileStream = fs.createReadStream(path, { encoding: 'utf-8' });
    fileStream.on('end', () => { cb(null); });
    fileStream.pipe(outputStream, { end: false });
}));

return outputStream;

but it's quite ugly.

[phated]

The way this was solved from the gulp perspective was attaching the fileStream as a contents property in an object and passing that through the original stream (in this case, filenameStream). I wouldn't recommend that approach since it has caused us all sorts of trouble; however, you can check out the implementation in vinyl-fs if you want.

[Ajaxy]

Thanks

[yocontra]

@Ajaxy This might be what you're looking for: https://github.com/teambition/merge2

Pseudo-code:

const outputStream = merge2();
const filenamesStream = gs(`${this.dirName}/*`);

filenamesStream.on('data', (path) =>
  outputStream.add(fs.createReadStream(path, { encoding: 'utf-8' }))
)

return outputStream;

Assuming you want to take every file in a folder and put the contents in a stream in sequence.