peterhaldbaek / delimiter-stream

Splits a stream into chunks separated by delimiter
MIT License
4 stars 3 forks source link

Empty data are skipped #5

Open avit opened 6 years ago

avit commented 6 years ago

I'm guessing this is an artifact of how Streams work, but when using on('data') events, or pipe to another transform function, no callbacks are invoked for empty data.

The effect is that doubled delimiters from the input such as two newlines (\n\n) are changed to a single newline in the output. (It looks like the empty string in between is not being processed.)

DelimiterStream = require('delimiter-stream');
rows = [];

linewise = new DelimiterStream('\n').on('data', (line) => { rows.push(line) });

linewise.write('one\ntwo\n\nfour\nfive');
linewise.end();

// expected 5, got 4
console.log(rows.length);
avit commented 6 years ago

One possible workaround might be an option to keep the delimiter on the output, trailing on the end of each chunk. Then, it would have to be up to the consumer to remove it if they want the data without it (e.g. String.trim() or other buffer operations).

peterhaldbaek commented 6 years ago

Yes, I think you are right. Empty data is probably treated as null preventing the Readable part of the stream from emitting the data event. There might be workarounds but I have not looked into it.