streamich / unionfs

Use multiple fs modules at once
https://www.npmjs.com/package/unionfs
The Unlicense
209 stars 24 forks source link

`createReadStream()` does not work with readline #92

Closed bytesnz closed 6 years ago

bytesnz commented 6 years ago

Testing a module that uses readline with a unionfs createReadStream does not work.

Error encountered:

(node:17269) UnhandledPromiseRejectionWarning: TypeError: input.on is not a function
    at new Interface (readline.js:184:11)
    at Object.createInterface (readline.js:63:10)

Test file:

const fs = require('fs');     
const readline = require('readline');
const mfs = require('memfs').fs;
const ufs = require('unionfs').ufs;
const util = require('util'); 

const contents = `this        
is a                          
test                          
file                          
`;                            

mfs.mkdirSync('/tmp');        

ufs.use(fs).use(mfs);         

const onLine = (type) => (line) => console.log(type + ': ' + line);
const onEnd = (type) => () => console.log(type + ':end');

const systems = { fs, mfs, ufs };

Object.keys(systems).forEach(async (key) => {
  const tfs = systems[key];   
  tfs.writeFileSync('/tmp/testfile', contents);
  const writeFile = util.promisify(tfs.writeFile);
  await writeFile('/tmp/testfile1', contents);

  const rl = readline.createInterface({
    input: tfs.createReadStream('/tmp/testfile')
  });                         

  rl.on('line', onLine(key)); 
  rl.on('end', onEnd(key));   
});

Actual file under test: https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/lib/file.js (https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/src/lib/file.ts)

Using files: https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/lib/file.test.js (https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/src/lib/file.test.ts) https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/test/lib/unionfs.js (https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/src/test/lib/unionfs.ts)

streamich commented 6 years ago

As of now unionfs does not properly support streams.

It can be implemented, but requires substantial work. Basically ReadStream and WriteStream classes as well as createReadStream and writeReadStream would need to be re-implemented inside unionfs and they would have to use unionfs for their operations. Basically the above mentioned methods need to be ported, similar how it is done here:

bytesnz commented 6 years ago

Would it be possible to just pass through to the createReadStream functions from the union-ed filesystems, eg go through the filesystems until one doesn't throw an error?