search

sighjs / sigh

multi-process expressive build system for the web and node.js, built using baconjs observables
209 stars 12 forks source link

Batch mode and end of stream event #47

Closed jmatsushita closed 8 years ago

jmatsushita commented 8 years ago

cc @joelanders

In relation to sighjs/sigh-process#2 we were wondering if there would be such a thing as a batch mode (which is currently what really happens when running sigh without flags). In this mode, you could send an event to notify that the stream has ended.

This would allow us to create a sigh-exec plugin which could decide what to do with the end of stream and for instance:

Not sure whether that should be an option in sigh-core or the core glob plugin? What do you think?

insidewhy commented 8 years ago

Streams can already be closed from a plugin and this event can be subscribed to from subsequent plugins in the pipeline. In watch mode I don't think streams would ever be closed so not sure how useful closing streams manually is.

jmatsushita commented 8 years ago

Err... So I'm not talking about watch mode here. Also I'm probably using Stream the wrong way. I meant the stream of events.

I'm also talking about when sigh is ran without flag (which I guess you could call batch mode). In this mode, if we want to execute a process (specifically a static site generator) then we need to wait until the stream is finished (i.e. files have all been processed) before we run the generator (ideally we'd use a streaming static site generator but I don't think this exists yet).

So it seems that we need upstream i.e. glob to tell us when all files have been processed in this scenario? (although we also need to think about multiple globs upstream!)

insidewhy commented 8 years ago

@jmatsushita glob already forwards all globbed files in its first event. I think maybe merge's new collectInitial option might help in cases where you have multiple glob streams being merged (been in the code/docs for just a few days)? In other cases maybe debounce?

jmatsushita commented 8 years ago

Ah ha! Yes it looks like collectInitial will do the trick. We'll try this tomorrow! (getting late in this timezone).

insidewhy commented 8 years ago

nippon desu?

jmatsushita commented 8 years ago

Hanbun dake. France-jin desu. :)

insidewhy commented 8 years ago

J'aime France mais pas Paris