search

NICTA / scoobi

A Scala productivity framework for Hadoop.
http://nicta.github.com/scoobi/
482 stars 97 forks source link

Provide toEphemeralStream method #290

Closed adelbertc closed 11 years ago

adelbertc commented 11 years ago

Iterable[_], which is what is returned by the groupBy method, does not have a scalaz Applicative instance (for good reason). If I want to get all pairs between two Iterable[_]'s ideally I would like to do

myIterable tuple myIterable

Using .toStream, the left side will still be realized - if we can unfold an EphemeralStream from the structure we can avoid that.

etorreborre commented 11 years ago

Wouldn't that be better to have Scalaz provide an Iterable[T] => EphemeralStream[T] function?

tonymorris commented 11 years ago

This function would be very buggy. Iterable[T] => EphemeralStream[T] how would it look?

etorreborre commented 11 years ago

I am closing this issue for now because as Tony wrote there is no function which could provide Iterable[T] => EphemeralStream[T]. I don't see how to implement length for example.

tonymorris commented 11 years ago

It is more that such a function is dangerous, mixing non-strict evaluation with side-effects. That danger is well beyond any reasonable benefit. On 09/09/2013 3:01 PM, "Eric Torreborre" notifications@github.com wrote:

I am closing this issue for now because as Tony wrote there is no function which could provide Iterable[T] => EphemeralStream[T]. I don't see how to implement length for example.

— Reply to this email directly or view it on GitHubhttps://github.com/NICTA/scoobi/issues/290#issuecomment-24041184 .