Handling failures

[rpominov]

We define Stream in a way that does't support any first-class failures handling: we only have a sink() function to which we can push stuff (usually success branch values). I wish to keep things that simple, because this simple streams idea is the core of this project. Basically the purpose of the project is to explore how far we can get with this very simple definition of streams. See Main idea.

Yet we can try and find a way to support failures/errors. One thing we should try is to wrap Maybe/Either/Validation in streams. When we need failures handling in a stream, we can use Stream<Either<E,A>> (or Maybe/Validation).

One problem we will have with this approach is boilerplate code, here is how it can be solved (pseudocode, with some imaginary Either and Async abstraction impl.)

// suppose we have two async operations that may fail, readFile and parseJson,
// and we want to chain them

// with failures handling in Future (just for comparison)
readFile(path).chain(parseJson)

// using Either manually
readFile(path).chain(either =>
  either.fork(
    () => Future.of(either),
    parseJson
  ))

// helper
const chainEither = fn => either =>
  either.fork(() => Future.of(either), fn)

// with helper
readFile(path).chain(chainEither(parseJson))

We can have similar helpers for map/filter etc.

Not yet sure if this approach with helpers will work in all cases, also I wonder if we could provide some helpers like this in basic-stream package.

I'd very appreciate any thoughts/ideas on this particular approach and on the failures handling in general.

[rpominov]

Related:

• https://github.com/ramda/ramda-fantasy/issues/73
• https://gitter.im/ramda/ramda?at=56a76e57aaae7a3a7593297f

[mickvangelderen]

Funny, I've also been toying with this idea. Working with Streams should be a lot easier.

[abuseofnotation]

Cool idea. However the proposed syntax bugs me a bit - what I would like to see in a stream library is a wrapper that handles errors natively, which I can use if I want to.

Handling errors is no biggie. I would code a custom Error handling mechanism.

Also what is the point of Maybe in a stream? If there is no value, I wouldn't call the callback, as opposed to calling it w/ "nothing".

Otherwise, I am researching this kind of stuff in an experimental monad transformer lib that I am building. Take a look if you want - I am experimenting w/ integrations with Folktale Future.

[rpominov]

However the proposed syntax bugs me a bit - what I would like to see in a stream library is a wrapper that handles errors natively, which I can use if I want to.

Yeah, I thought about it too. There is even two ways how we could do this:

• As a wrapper similar to Fantasy Land wrapper
• As another set of independent functions (map, filter, chain etc.) that operate not on Stream<A> but on Stream<Either<E,A>>

What bugs me about this approach is that we may end up with something similar to libs like Most or RxJS, and then the question is "why not simply use them?".

Also what is the point of Maybe in a stream? If there is no value, I wouldn't call the callback, as opposed to calling it w/ "nothing".

Good point, I guess I didn't thought too much about it :)

Otherwise, I am researching this kind of stuff in an experimental monad transformer lib that I am building. Take a look if you want - I am experimenting w/ integrations with Folktale Future.

Yeah, I would like to take a look. Do you have a link?

[abuseofnotation]

Yeah, its https://github.com/boris-marinov/monad-transformers

[rpominov]

@boris-marinov This is very interesting stuff. We probably want to keep things much simpler in this project. Although I'm still very new to monad transformers, discovered them very recently and still in the process of understanding them. Maybe we'll be able to create some very simple implementations of those ideas.

[rpominov]

Did an example of solving async-problem with data.validation for representing failures.

https://github.com/rpominov/basic-streams/tree/master/examples/async-problem

[rpominov]

Hm, one way we could go is to add support of FL types wrapped inside basic-streams. We could add methods like:

mapInner = (A => B) => Stream<Functor<A>> => Stream<Functor<B>>
chainInner = (A => Chain<B>) => Stream<Chain<A>> => Stream<Chain<B>>
combineArrayInner = Stream<[Applicative<A>]> => Stream<Applicative<[A]>>
etc.

Not sure how many use cases of working with Stream<Either|Validation> we could cover with that general approach though...

[Avaq]

@rpominov The idea of monad transformers is that your monad type knows how to generate these "methods" when given another monad type. Many FL types implement a static .T() which takes another Monad type, and returns a new Monad type on which the chain looks a lot like your chainInner, the map a lot like your mapInner, etc.

For example; members of Stream.T(Either) could be thought of as members of Stream<Either>, where every monadic transformation is automatically lifted into the inner monad.

I'm not sure if it'd be possible to create a transform function like that for Stream though, for the same reasons people haven't found a way to get it to work with Futures in ramda#73. The monad-transformers lib seems to take a different approach (from the A.T(B)), which might be worth looking into if it solves the async transformers issues!

[rpominov]

@Avaq I'm still trying to figure out whether I have the same idea as transformers or a bit different one. Just to illustrate how confused I'm at the moment, I have in mind four variations of chain method. The first one is just normal Stream's chain, and the last is what we should have on Stream.T(Inner) probably. The other two is something in between.

// Normal chain method (we flatten Stream<Stream> to Stream)
chain = (Inner<A> => Stream<Inner<B>>) => Stream<Inner<A>> => Stream<Inner<B>>

// Apply chain to inner type (we flatten Inner<Inner> to Inner)
chainInner = (A => Inner<B>) => Stream<Inner<A>> => Stream<Inner<B>>

// Apply chain to Stream, but using value wrapped to inner type 
// (we flatten Stream<Stream> to Stream, but with a function that operate on type inside Inner)
chainOuter = (A => Stream<B>) => Stream<Inner<A>> => Stream<Inner<B>>

// chainInner and chainOuter composed somehow...
chainBoth = (A => Stream<Inner<B>>) => Stream<Inner<A>> => Stream<Inner<B>>

Note: the last two can't be generic (for any monad as Inner), we need to use low level methods of Inner type in order to implement them.

[rpominov]

Hm, replaced chainOuter with chainBoth in async-problem example and it became simpler. Seems like a move in the right direction: https://github.com/rpominov/basic-streams/commit/5e622cfb545127c3f37b0c4ef5442a2869da16d6 (the function is called chainV in it)

[Avaq]

Aren't chainInner and chainOuter just compose(map, chain) and something like compose(chain, sequence) respectively? I played around a bit here and it seems like it.

[rpominov]

@Avaq You are totally right! And I think I managed to implement generic chainBoth given inner type has chain and sequence. It a bit messy now, but it's all there: https://github.com/rpominov/basic-streams/blob/master/examples/async-problem/index.js (I'll rewrite it with FL wrapper for BS, which hopefully make it cleaner)

Still a lot to do, but I think we did a huge progress today :)

[abuseofnotation]

I'm not sure if it'd be possible to create a transform function like that for Stream though, for the same reasons people haven't found a way to get it to work with Futures in ramda#73.

If you want to handle failures in Stream, what you need is EitherT, not StreamT.

[rpominov]

@boris-marinov So you mean inner type should be "controller" so to speak? In ramda-fantasy outer type is used as "controller", and in your lib it's inner type.

This is interesting though, if we unable to create Future(Either) with Future as controller, maybe it's possible to do with Either as controller.

There are couple problem though:

1. The type of this lib is Stream, so if Either is controller this transformer will be out of scope of this lib.
2. Don't we loose all the rest of custom functionality of Stream type in this case? Like filter, chainLatest, takeWhile, etc.?

Also I still not sure that what I'm trying to do is monad transformer, it's certainly similar but still different. First of all I plan to use not only Monad interface of inner type, but whole range of interfaces from fantasy-land (we'll need Applicative and Traversable for example).

[rpominov]

I'm not sure if it'd be possible to create a transform function like that for Stream though

Btw, we already solved that, I think, except the inner type must also be Traversable.

[abuseofnotation]

A monad transformer is both inner and outher.

Looks like this:

EitherT ( Stream (Either) )

from https://en.wikibooks.org/wiki/Haskell/Monad_transformers

The chained functions in the definition of return suggest a metaphor, which you may find either useful or confusing. Consider the combined monad as a sandwich. This metaphor might suggest three layers of monads in action, but there are only two really: the inner monad and the combined monad (there are no binds or returns done in the base monad; it only appears as part of the implementation of the transformer). If you like this metaphor at all, think of the transformer and the base monad as two parts of the same thing - the bread - which wraps the inner monad.

1. This is up to you to decide.
2. Yes, unless the transformer is a custom one.

[rpominov]

So yeah, if we own both types and know about custom methods of both types, everything becomes easier. But this is basically how Rx/Bacon/Most/Kefir/etc work, we have a lot of custom implementations with custom types. What I would like to achieve is that any type with certain Fantasy-Land interfaces could be used as inner without lots of boilerplate code.

[abuseofnotation]

Are you sure your inner type has to be a monad then? Isn't it enough for it to be a functor (so it can hold a value or an error), with an identity operation (so you can compose error-handling stream with a basic stream).

[rpominov]

I started working on a new wrapper similar to Fantasy Land wrapper but that wraps Stream<T<A>> where T supposed to implement certain FL methods.

https://github.com/rpominov/basic-streams/blob/master/src/fantasyT.js

Also updated async-problem to use that type.

So far I added to that wrapper only methods that were needed for the example. Very curious how many methods can be implemented in it. Gonna continue exploration soon.

@boris-marinov

Are you sure your inner type has to be a monad then?

Yeah, we need it to have a monadic chain to implement our composed chain see fantasyT.js#L25-L34, but it also has to be a functor (have map) and traversable (have sequence)

[rpominov]

Not sure how to name this thing. Currently it's named StreamT, but it isn't quite a monad transformer.

[rpominov]

Rearranged everything a bit, and now it looks like this const StreamV = Stream.Compose(Validation), I like it :)

[rpominov]

Finally finished the composition example. I've made BasicStream to be a Static Land type. And implemented composition in Static Land types instead of Fantasy Land. So I first create a SL type for data.validation and then compose two SL types like so:

const StreamV = Stream.composeWithInnerType(SValidation)

Check it out. Although this is all still just a POC, as well as this entire repo.

[rpominov]

As part of process of making basic-streams production ready I've removed this experimental stuff for now. And don't have plans on working in this area in near future, so closing for now.