caolan / highland

High-level streams library for Node.js and the browser
https://caolan.github.io/highland
Apache License 2.0
3.42k stars 145 forks source link

Easily output normal node streams #129

Open mpj opened 9 years ago

mpj commented 9 years ago

When I'm writing modules, I'd like the API I expose to return normal node streams instead of highlands special ones. Some kind of .toNodeStream() would be nice. This would also be solved if highland streams subclassed node streams.

greelgorke commented 9 years ago

which part of the core API do you miss on the Highland stream? Or better, why do you need it really? :)

mpj commented 9 years ago

@greelgorke I don't miss anything, I just don't want to my modules that use highland to provide a predictable API. It's how when you use a DOM manipulation library that uses jQuery behind the scenes, you don't want it's API to return jQuery objects, you just want plain DOM elements.

greelgorke commented 9 years ago

@mpj i understand that. my question though targets to the fact, that all highland streams support some of core api already, most importantly the pipe protocol.

though there is no .read() on highland, which could be wrapper around pull. insofar this would make a nice pr :) if i only had time

stephen-bartell commented 9 years ago
return _()
  .pipe(new Stream.PassThrough({ objectMode:true }))

is what i do

greelgorke commented 9 years ago

this is also a way, sure, but it's then just a node.js stream. After reading @mpj's last post again, i have to say, that the comparison with jQuery isn't that good. jQuerys aim was always to handle and hide DOM, so there is this clear focus, and it legit to return DOM elements here.

Highland.js isn't a wrapper around Node.js Streams. If you look at the first paragraphs, Array is first class predecessor, and Node.js is the second, which you also can see in the api.

But with saying that i'm still convinced to have a full complient api with node streams.

yangit commented 9 years ago

Here it the non compliant part: Node streams can be consumed multiple times w\o extra effort. If I'm passing my stream to other lib which later does stream.pipe(a) stream.pipe(b) Then I have an exception.

One example would be GulpJS

NathanRSmith commented 9 years ago

Thanks for the tip @stephen-bartell. That solved a problem I was having. I was trying to pipe a highland stream into a REST PUT request using the request.js module but it was only sending the first ~15kb depending on some settings I could tweak. I'm not exactly sure what was wrong, but as soon as I put the intermediate .pipe(new Stream.PassThrough({ objectMode:true })) in it started sending the whole stream.

Even though this method works, it still feels like a hack and it would be nice if there was a more concise way to get back to regular node streams. Or at least info in the docs discussing how to send data to non- objectMode=true streams.