dominictarr / through

simple way to create a ReadableWritable stream that works
Other
670 stars 64 forks source link

Pause doesn't affect write #23

Open azer opened 10 years ago

azer commented 10 years ago

In a case, I need to pause streaming until another part of the program is ready. I call the pause function, but through keeps calling the write function, which needs to wait until I call resume.

For example:

    var paused = true
    var ts = through(function () {
         console.log('paused?', paused)
     }).pause()

     fs.createReadStream('foobar').pipe(ts)

     setTimeout(function () {
         paused = false
         ts.resume()
     }, 1000)

Will output:

paused? true

Is it the intended behavior?

dominictarr commented 10 years ago

the paused variable you have is nothing todo with through (it's just a variable you have created in your scope. instead use ts.paused to get the current pause state of the stream).

azer commented 10 years ago

thanks for the tip. so we can change that code to:

var through = require('through')
var fs = require('fs')

var ts = through(function () {
  console.log('paused? ', ts.paused)
}).pause()

fs.createReadStream(__filename).pipe(ts)

setTimeout(function () {
  ts.resume()
}, 1000)

Which will output:

paused?  true
dominictarr commented 10 years ago

yes - that is not unexpected. the problem with classic node streams is that they do not propagate the pause signal until they are written to, so you have to get a write before the upstream knows it's time to pause.

but the real question is what are you actually trying to do and why?

azer commented 10 years ago

I need it for stream-format

It takes an object of streams, and a template to render. It needs to wait until the context is loaded. Here is an example:

format = require('stream-format')

f = format({ foo: fs.createReadStream("./foo"), bar: fs.createReadStream("./bar') })
f.pipe(process.stdout)

fs.createReadStream("./template").pipe(f)

You can check out the tests at: https://github.com/azer/stream-format/blob/master/test.js#L56

And the way implement it right now is using https://github.com/azer/pause-function on write and done functions that I pass to through

dominictarr commented 10 years ago

This is a great time to write documentation, I'm not reading your tests - that is just giving me too much work.

does the template really need to be a stream? usually templates are very finite. I doubt that it needs to be streaming... you probably need to buffer the whole template before you can use it anyway. what do you mean by 'context must be ready'? does that mean the template hasn't streamed?