Reducer Composition with Effects in JavaScript

[gaearon]

Inspired by this little tweetstorm.

Problem: Side Effects and Composition

We discussed effects in Redux for ages here but this is the first time I think I see a sensible solution. Alas, it depends on a language feature that I personally won’t even hope to see in ES. But would that be a treat.

Here’s the deal. Redux Thunk and Redux Saga are relatively easy to use and have different pros and cons but neither of them composes well. What I mean by that is they both need to be “at the top” in order to work.

This is the big problem with middleware. When you compose independent apps into a single app, neither of them is “at the top”. This is where the today’s Redux paradigm breaks down—we don’t have a good way of composing side effects of independent “subapplications”.

However, this problem has been solved before! If we had a fractal solution like Elm Architecture for side effects, this would not be an issue. Redux Loop implements Elm Architecture and composes well but my opinion its API is a bit too awkward to do in JavaScript to become first-class in Redux. Mostly because it forces you to jump through the hoops to compose reducers instead of just calling functions. If a solution doesn’t work with vanilla combineReducers(), it won’t get into Redux core.

Solution: Algebraic Effects in JavaScript

I think that what @sebmarkbage suggested in this Algebraic Effects proposal is exactly what would solve this problem for us.

If reducers could “throw” effects to the handlers up the stack (in this case, Redux store) and then continue from they left off, we would be able to implement Elm Architecture a la Redux Loop without the awkwardness.

We’d be able to use vanilla combineReducers(), or really, any kind of today’s reducer composition, without worrying about “losing” effects in the middle because a parent didn’t explicitly pass them up. We also would not need to turn every reducer into a generator or something like that. We can keep the simplicity of just calling functions but get the benefits of declaratively yielding the effects for the store (or other reducers! i.e. batch()) to interpret.

I don’t see any solution that is as elegant and simple as this.

It’s 2am where I live so I’m not going to put up any code samples today but you can read through the proposal, combine it with Redux Loop, and get something that I think is truly great.

Of course I don’t hold my breath for that proposal to actually get into ES but.. let’s say we could really use this feature and Redux is one of the most popular JavaScript libraries this year so maybe it’s not such a crazy feature as one might think at first :smile: .

---

cc people who contributed to relevant past discussions: @lukewestby @acdlite @yelouafi @threepointone @slorber @ccorcos

[evenstensberg]

I'd love this with a twist. Having the previous state to automatically go into an array through object.Assign/other invoke methods and maybe have them to be pure reducers/actions if you want them to based upon needs. Say you have a() -> 5 and b -> 7 and want c = a() + b(); ... If you want to use a later, you can reference the function as Sebastian proposed and later use it in another call by doing it from the array of yields. Sorry for explaining this very. Badly and with no code highlighting. 4 am and writing this at my iPad, will clarify tomorrow if needed

[gaearon]

Not sure what you mean, code would definitely help :smile: . I’ll try to put up some examples too when I find some time.

[lukewestby]

Agreed on the awkwardness of composing Effects in middle reducers. The Elm compiler plays a huge role in making it doable with elm-effects. We've already begun to experiment with algebraic effects at Raise by co-opting generators to yield effects or yield spawn() other generators. I think it's about as close an approximation as we've been able to come up with without inventing syntax and writing a babel plugin. I really like @sebmarkbage's point about not needing three different execution style notations for async vs generator vs plain, and we've approximated that by just saying everything is a generator because anything might want to yield an effect at some point. We've been successful so far using it for request handlers in Node, but I'm not really sure how it would look when combined with redux-loop. I think this might be more of an evolution from redux-loop than a combination with it, what do y'all think?

[acjay]

Interesting! So would there be a new concept of effect handlers?

Related reading: http://blog.paralleluniverse.co/2015/08/07/scoped-continuations/

Redux-loop seems to basically be the monadic idea. A reducer in that sense is the argument to flatMap. The delimited continuation model seems similar to that author's scoped continuations.

One thing that seems weird is the use of continuations. Is it too much power to have values injected back into the continuation? Maybe it's just grating against my Redux guide training, but suddenly it seems like reducers are going to get a lot more complicated, with fetching happening in-line. But I suppose that complexity has to live somewhere.

Another thought: with monadic style, you can test each phase independently, but you kind of lose that ability if you have one long thread of control.

I haven't ever tried to compose separate Redux-based components, so my thoughts are all from the perspective of a monolith. More thoughts on how this would make Redux apps more composable would be helpful, to me at least.

[eloytoro]

Hasn't anyone considered using something such as co to simulate an "effect like" operation? It looks like it could fill the current void for an effect syntax

[acjay]

I use co, but I'm not sure what you mean. It's basically just an option for getting async/await with generators. From the use I'm familiar with, it's not so much an effect executor as a way to work with promises. You still have to fire off the effect yourself, which is impure. In other words, you want to be able to yield some value that an effect handler will use to trigger a fetch, and inject the returned value back. But with co, you'd execute the fetch yourself and yield the resulting promise to resolve it. That's not decoupled in the same way.

[eloytoro]

While co is designed to handle promises there should be a way to do the handling of synchronous, pure effect spawning within reducers using a similar approach On Mar 17, 2016 11:45 PM, "Luke Westby" notifications@github.com wrote:

^ Totally. Promises shouldn't show up anywhere in the reducer since they necessarily represent the execution of an async function. Doesn't always mean a side-effect occurred, but it can and usually does and that's what we're here to avoid.

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/reactjs/redux/issues/1528#issuecomment-198200184

[Antontelesh]

We can experiment with the syntax like Elm provides. Reducer could return not only the next state but also an effect. This effect could be represented as IO monad for example.

function reducer (state, action) {
  if (actionRequiresEffect(action)) {
    return [nextState, IO.of(function () {
      // here is the effect code
    })];
  }
}

[guigrpa]

The throwing effects idea looks promising, but I have some concerns (even if such a mechanism eventually found its way into ES):

• Maybe the solution should not use the return channel in Sebastian's proposal (injecting values back to the reducer, even asynchronously), since IMHO that would make reducers impure or at least more difficult to reason about. The store should immediately call the continuation with no arguments whenever it catches an effect, and then process the effect asynchronously.
• If effect handling would be built into core Redux, it would probably make it much more complex, having to handle execution of many types of effects (sync, async, parallel, serial, cancellable, etc.), just as they are now processed by redux-saga. On the other hand, if this would be left to a store enhancer we might end up with deferred continuations and more complex reducers.

[ganarajpr]

I implemented something quite similar to this idea. This was something that I had to come up with to solve the redux-elm-challenge.

You can look at my solution here. https://github.com/slorber/scalable-frontend-with-elm-or-redux/tree/master/localprovider-redux-saga-ganaraj

The core of the idea is that the Provider given by React-redux is not something that has to be a single entity up the component chain. You could technically have multiple child stores down the chain but you could handle the actions they emit anywhere up the chain. This is an idea quite similar to Angular 2's child injectors.

This is the entire code of the LocalProvider.

import React, {createElement, Component} from 'react';
import { createStore, applyMiddleware } from 'redux';

const localState = (component, localReducer, mws=[]) => {

  class LocalProvider extends Component {
    constructor(props, context) {
        super(props, context);
        this.parentStore = context.store;
        this.listener = this.listener.bind(this);
        this.localStore = createStore(localReducer, 
            applyMiddleware.apply(null, [this.listener, ...mws]));
        this.overrideGetState();        
    }

    overrideGetState(){
        const localGetState = this.localStore.getState;
        this.localStore.getState = () => ({
                ...this.parentStore.getState(),
                local: localGetState()
            });
    }

    getChildContext() {
        return { store: this.localStore };
    }

    listener() {
        return (next) => (action) => {
            let returnValue = next(action);
            this.parentStore.dispatch(action);
            return returnValue;
        };
    };

    render() {
        return createElement(component, this.props);       
    }
  }  
  LocalProvider.contextTypes = {store: React.PropTypes.object};
  LocalProvider.childContextTypes = {store: React.PropTypes.object};
  return LocalProvider;
};

export default localState;

Currently this has the disadvantage that its not possible to serialize the child/ local stores. But this is something that is easy to handle if we had some support from react itself, if it exposed the _rootNodeId OR if we had a library that provided the path to the current node from the root node. I believe this is a relatively easier problem to solve eventually. But it would be quite interesting to know what others think about a solution like this.

[tomkis]

We also realized that the lack of fractability in Redux is a bit limiting for us to build really scalable and complex application.

About a half year ago we were experimenting with changing the shape of reduction into Pair<AppState,List<Effects> which is basically the same like redux-loop is doing and came to exactly the same conclusion like @gaearon. It's simply too difficult to mix effectful and effectless reducers and composition becomes very clumsy.

Frankly, sometimes it's quite clumsy even in Elm because the programmer is still responsible for unwrapping the Pair manually and passing everything appropriately to sub-updaters.

Therefore we've built https://github.com/salsita/redux-side-effects which is at least "somehow" trying to simulate the Continuations with Effect handlers approach. The idea is very simple, if reducer was generator then you could simply yield side effects and return mutated app state. So the only condition is to turn all your reducers into function* and use yield* whenever you want to compose them.

function* subReducer(appState = 0, { type } ) {
  switch (type) {
    case 'INCREMENT':
       yield () => console.log('Incremented');

       return appState + 1;
    case 'DECREMENT':
       yield () => console.log('Decremented');

       return appState - 1;
    default:
       return appState;
  }
}

function* reducer(appState, action) {
  // Composition is very easy
  return yield* subReducer(appState, action);
}

Reducers remain pure because Effects execution is deferred and their testing is a breeze.

And because I am also author of redux-elm I tried to combine those two approaches together and the result? I've ported the Elm architecture examples into Redux and was really surprised how nicely it works, there's no even need for unwrapping the reduction manually

The only drawback so far is that yield* is not automatically propagated in callbacks, therefore if you want to map over some collection and yield side effects inside the mapper function then you can't. The workaround is to map over the collection and yield list of effects. Or use generator "friendly" version of map.

We've been using this approach in production for almost a half year now. Seems like the perfect fit for us in terms of highly scalable architecture for non-trivial applications even in larger teams.

[slorber]

@gaearon the problem you'd like to solve seems quite similar to the one here: scalable-frontend-with-elm-or-redux

The problem is not clearly solved yet weither it's Elm or Redux..., and even Elm fractability alone does not help that much in my opinion.

Fractal architectures often follow your DOM tree structure. So fractal architecture seems perfect to handle view state. But for which reason exactly the "effects" should also follow that structure? Why should a branch of your tree yield effects in the first place?

I really start to think it's not the responsability of a branch of your view tree to decide to fetch something. That branch should only declare what has happened inside it and the fetch should happen from the outside.

Clearly I don't like at all the idea of view reducers yielding effects. View reducers should remain pure functions that have a single responsability to compute view state.

[yelouafi]

@gaearon If I understand what you're trying to solve is the boilerplate involved when adopting an Elm approach to side Effects. I'm not yet familiar with delimited continuations but from @sebmarkbage proposal what you're proposing could be something like this

function A(state, action) {
  // ... do some stuff
  perform someEffectA
  // ... continue my stuff
  return newState;
}

function B(state, action) {
  // ... do some stuff
  perform someEffectB
  // ... continue my stuff
  return newState;
}

rootReducer = combineReducers({ A, B })

and then somewhere the store do something like

do try {
  let nextState = rootReducer(prevState, action)
  prevState = nextState
  runAllEffects()
} 
catch effect -> [someEffectA, continuation] {
  scheduleEffect(someEffectA)
  continuation() // resume reducer A immediately
  /* 
    after reducer A finishes, continue to reducer B (?)
    What happens if reducer B throws here
    Or perhaps the do try/catch effect should be implemented
    inside combineReducers
  */
}

catch effect -> [someEffectB, continuation] {
  scheduleEffect(someEffectB)
  continuation() // resume reducer B immediately
}

Maybe I missed something with the above example. But I think there are some issues which still has to be answered

• After catching some effect, the store has to schedule the effect for the execution later, because the reducers should be resumed immediately (reducer has still to be synchronous). Then there will be still the issue of managing control flow (long running transactions) using the state machine approach
• How would it work if a parent reducer should (say a reducer for a router component since we aim for fractability here) catch effects from reducers of nested components. Does it have to replicate the Store catch Effect/invoke the continuation mechanism ? And if yes, wouldn't the flow of thing be a little complicated to reason about here (where the continuation frame begins and where it ends when we're in the middle of a reducer chain)
• And there are also the issues related to Component decoupling raised by @slorber in his Elm challenge,

From a more conceptual POV, AFAIK continuations (call/cc) are about providing a pretty low level way of decomposing/manipulating the flow of a program. I haven't looked in detail into delimited continuations but I guess they provide more flexibility by making possible to capture the continuation from 2 sides (having the continuation return a value). This gives far more power but also would make the flow pretty hard to reason about. My point is that for what you're aiming to achieve, it seems to me that you're using a too much heavy weapon :)

I agree that redux lacks fractability, which Elm achieves by having a different way of decomposing/recomposing things (Model/Action/Update) But IMO the Elm way inevitably introduces the boilerplate of passing things down/up. And with the Effect approach it creates even more boilerplate because Elm reducers (Update) have to unwrap/rewrap all intermediate values (State, ...Effects) that bubbles up the Component tree, without mentioning the wrapping of the dispatch functions when the action tunnels down. (So I'm not sure how Elm language provides an advantage here over redux-loop besides type checking, because the boilerplate seems inherent to the approach itself).

IMO fracttability could be achieved by taking the Store itself as a Unit of Composition. Dont know exactly how this would be concretely but I think this would make Redux apps composable without being opinionated on a specific approach for Side Effects and Control Flow. This way the Store has to worry only about actions and nothing elses

[tomkis]

@gaearon the problem you'd like to solve seems quite similar to the one here: scalable-frontend-with-elm-or-redux

Your point is certainly correct, that's definitely a drawback of fractable architecture and there still are some techniques to solve that, like for example Action pattern matching, which as you may protest breaks encapsulation. On the other hand, from my experience this is a quite rare use case because mostly you only need direct parent <-> child communication which can be accomplished in very elegant way using The Elm Architecture.

I really start to think it's not the responsability of a branch of your view tree to decide to fetch something. That branch should only declare what has happened inside it and the fetch should happen from the outside.

Why to put such a constraint to only solve fetch? It's about side effect in general, while fetch may potentially not be tied to the Component, there certainly are different side effects which are Component specific (DOM side effects)

Clearly I don't like at all the idea of view reducers yielding effects. View reducers should remain pure functions that have a single responsability to compute view state.

I disagree, imagine you have a list of items which you may somehow sort using Drag n Drop. Your reducer is responsible for deriving the new application state, therefore the reducer is authoritative entity to define the logic. Persisting sort order is just a side effect of the fact!

In other words, saying that Event log is a single source of truth is utopia because you still need derived state to perform side effects anyway.

[tomkis]

@yelouafi

Then there will be still the issue of managing control flow (long running transactions) using the state machine approach

Yes, the Elm Architecture is missing that piece and that's exactly where I believe Saga is very useful and needed. On the other hand using redux-saga or redux-saga-rxjs for side effects is just a side effect of the pattern itself, it solves long running transactions and Request <-> Response is long running transaction. We could say as well that Sagas should be pure and we could combine those two approaches, which IMO seems most reasonable to me.

People misunderstood concept and usefulness of Sagas

const dndTransaction = input$ => input$
  .filter(action => action.type === Actions.BeginDrag)
  .flatMap(action => {
    return Observable.merge(
      Observable.of({ type: Actions.Toggled, expanded: false }),
      input$
        .filter(action => action.type === Actions.EndDrag)
        .take(1)
        .map(() => ({ type: Actions.Toggled, expanded: true }))
    );
  });

This is still a very useful Saga for solving long running transaction, yet it's totally effectless.

[jlongster]

As I said on twitter, you don't need continuations here. Continuations are really powerful, and are extremely useful when you need to control the execution flow. But reducers are restricted and we know exactly how they act: they are fully synchronous, always. Sebastian's proposal is great but it's really just a more powerful generator: instead of being a shallow yield, it's a full deep yield.

It's neat because you can control the execution and do stuff like this. Here we don't even call the continuation! But we don't need this with reducers.

try {
  otherFunction();
} catch effect -> [{ x, y }, continuation] {
  if(x < 5) {
    continuation(x + y);
  }
  console.log('aborted');
}

While we could still force reducers to be synchronous (like we can with generators), there is a still a debugging cost. This is probably the main reason TC39 has been against deep yields so far: they are harder to reason about, and debugging can get painful as things jump around (they also like the sync/async interface dichotomy). But we're not going to see this native any time soon, so your only hope is compilation, but compiling this sort of stuff involves implementing true first-class continuations completely and the generated code is really complex (and slow-ish).

I think it goes against the philosophy of reducers as just simple synchronous functions. You realize that with his proposal you will have to always call reducers with a try/catch effect, right? They are no longer simple functions that can be called. This is because perform only works inside the context of a try/catch effect, just like yield only works inside of function*. So in your tests calling a reducer would always be:

try {
  let state = reducer(state, action);
} catch effect -> [effect, continuation] {
  continuation();
}

Even if the reducer has no effects.

Let me show you what I was talking about on twitter. I'm not saying this is a good idea (it may be) but if want to "throw" effects inside reducers and "catch" them outside, all while retaining the existing interface, here's what you do.

You can do this with dynamic variables. JS doesn't have them, but in simple cases they can be emulated.

First you create 2 new public methods that reducer modules can import:

// Public methods that are statically imported

let currentEffectHandler = null;
function runEffect(effect) {
  if(currentEffectHandler) {
    currentEffectHandler(effect);
  }
}

function catchEffects(fn, handler) {
  let lastHandler = currentEffectHandler;
  currentEffectHandler = handler;
  let val = fn();
  currentEffectHandler = lastHandler;
  return val;
}

We're basically making runEffect a dynamic variable. catchEffects may override this variable and run functions in different contexts. We're leveraging the JS stack to create a stack of effect handlers.

Now let's create a sample store. This does what @gaearon initially described: just pushes effects onto an array. The top-level uses catchEffects itself to gather up effects across the entire call.

let store = {
  dispatch: function(action) {
    let effects = [];
    let val = catchEffects(() => reducer1({}, action), effect => {
      effects.push(effect);
    });
    console.log('dispatched returned', val);
    console.log('dispatched effected', effects);
  }
}

Now some reducers:

// Sample reducers. These would do:
// `import { runEffect, catchEffects } from redux`

function reducer2(state, action) {
  if(action.x > 3) {
    runEffect('bar1');
    runEffect('bar2');
    runEffect('bar3');
  }
  return { y: action.x * 2 };
}

function reducer1(state, action) {
  runEffect('foo');
  return { x: action.x,
           sub1: reducer2(state, action) };
}

Both reducers initiate some effects. Here's the output of store.dispatch({ type: 'FOO', x: 1 });:

dispatched returned { x: 1, sub1: { y: 2 } }
dispatched effected [ 'foo' ]

And here's the output of store.dispatch({ type: 'FOO', x: 5 });:

dispatched returned { x: 5, sub1: { y: 10 } }
dispatched effected [ 'foo', 'bar1', 'bar2', 'bar3' ]

Reducers themselves can use catchEffects to suppress effects:

function reducer1(state, action) {
  runEffect('foo');
  return { x: action.x,
           sub1: catchEffects(() => reducer2(state, action),
                              effect => console.log('caught', effect))};
}

Now the output of store.dispatch({ type: 'FOO', x: 5 });:

caught bar1
caught bar2
caught bar3
dispatched returned { x: 5, sub1: { y: 10 } }
dispatched effected [ 'foo' ]

EDIT: Had some copy/pasting errors in the code, fixed

[yelouafi]

another way of doing could be something like the Context concept of React. If we suppose that reducers are always non-method functions we can leverage the func.call method to call them within a provided context. Throwing an effect would be equivalent to invoking a this.perform(...) where this is the current context.

function child(state, action) {
  //... do some stuff
  this.perform(someEffect)
 // ... continue
 return newState
}

function parent(state, action) {
  //... do some stuff
  this.perform(someEffect)
  return {
    myState: ...
    childState: child.call(this, state.childState,action)
  }
}

Similarly, catching effects from parent reducers could be done by having parents override the context passed to children

function parent(state, action) {
   const context = this
  //... do some stuff
  const childContext = context.override()
  const childState = child.call(childContext, state.childState, action)
  if(someCondition) {
    context.perform(childContext.effects)
  }
  return {
    myState: ...
    childState
  }
}

We can also test the reducers simply by passing them a mock context then inspecting the mock context after they return

[ganarajpr]

@yelouafi Kind of very similiar to what I am doing in LocalProvider - see a few comments above.

[jlongster]

Yes, React's context is similar to dynamic variables. That changes the signature of reducers though, you can't just call them as normal functions.

[lukewestby]

It would helpful for me at least for API ideas to include both usage and testing code examples. My guess from experience and intuition is that any test for a generator-based solution would look a lot like a test for a saga. What would tests look like using either the dynamic variable approach or the context approach?

[jlongster]

@lukewestby

// Test that a reducer returns the right state (literally no difference)
let state2 = reducer(state1, action);
assertEquals(state2.x, 5);

// If you want the effects, use `catchEffects` to get them
let effects = [];
catchEffects(() => reducer(state1, action), effect => effects.push(effect));
assertEquals(effects.length, 3);

Honestly this came out better than expected, I don't know how you could get much simpler and keep the existing interface 100%.

(and my post above has several code examples)

[jlongster]

@yelouafi Also, your solution doesn't work across arbitrary frames. If a reducer calls out to any helper methods the this context is lost. I'd like to see this go away from JS, and I don't think we should use it.

[tomkis]

@lukewestby yes:

function* reducer(appState = 0,  { type }) {
  switch (type) {
  case 'INCREMENT':
    yield effect(Logger.log, 'Incremented');
    return appState + 1;

  case 'DECREMENT':
    yield effect(Logger.log, 'Decremented');
    return appState - 1;

  default:
    return appState;
  }
}

it('should increment value and log', () => {
  const iterable = reducer(42, { type: 'INCREMENT' });

  assert.deepEqual(iterable.next().value, effect(Logger.log, 'Incremented'));
  assert.equal(iterable.next().value, 43);
});

it('should decrement value and log', () => {
  const iterable = reducer(42, { type: 'DECREMENT' });

  assert.deepEqual(iterable.next().value, effect(Logger.log, 'Decremented'));
  assert.equal(iterable.next().value, 41);
});

[acjay]

@jlongster Great outline of a solution. I've been considering a similar architecture for allowing React components to "yield up" other data besides VDOM.

It seems like in all these cases, we're basically looking for a writer monad -- (well, if you discount the proposed ability to inject data back into your scope). The basic concept is that you have a regular function that returns some core data, but you also want it to return some auxiliary data too. So you wrap them together. As you compose many of these functions, you provide the way in which pieces of core data should be combined to form a new unitary piece of core data, and the auxiliary data is simply concatenated as a collection. That concept of automatically handling the additional context in some natural way is the hallmark of a monad. Some top-level machinery decides what to do with your single piece of core data and collection of auxiliary data.

It's funny, because these variations are the same things I considered when writing react-redux-controller. There are basically 3 solutions pushing/pulling auxiliary data from a scope: monads (wrappers for your data), yield, and this. There are also closures, as beautifully illustrated here, but they are restricted to lexical scope. Provided that's a limitation you can live with, it's also a valid way to go. I think the downside there is that now you have to hack the module system if you want to test.

[yelouafi]

@jlongster

Yes, React's context is similar to dynamic variables. That changes the signature of reducers though, you can't just call them as normal functions.

You're right, you'll have to always use the method form because a parent can't make assumptions about a child reducer (if it needs this or not)

Also, your solution doesn't work across arbitrary frames. If a reducer calls out to any helper methods the this context is lost.

Again it's true.

But anyway I'm not really fond of melting down code that computes the view state with the code that drives the operations. I think it's good to keep the 2 totally separated.

If I wanted a pure/memory-less/state-machine approach for control flow (e.g. to have time travel and hot reload for my control flow) i'd consider having a separate reducer tree for effects (And by reducer tree, I don't mean a tree of states, i.e. all effect reducers should get the top state of the locally scoped component).

[lukewestby]

I really like @jlongster's solution so far. I'm going to see if I can implement it as a branch of redux-loop and report back with findings!

[lukewestby]

Seems to work really well after a first pass https://github.com/raisemarketplace/redux-loop/blob/deep_effect_collection/test/index.test.js#L35-L47

[jlongster]

@lukewestby: Very cool! Excited to see if it works out. I agreed with @yelouafi that I generally like keeping side effects outside of reducers, however this could prove useful in certain awkward situations. I've been meaning to study redux-sagas so I wonder how this whole thing can fit together.

I'll be keeping an eye on your experiements in redux-loop!

Note that my code was very much a prototype. It might need a few tweaks, for example a finally around the fn call in catchEffects:

try {
  let val = fn();
} finally {
  currentEffectHandler = lastHandler
}

Additionally, if you want to support re-"throwing" effects in a handler, you'll need to keep a stack of handlers instead of using the JS stack. That way you can run currentEffectHandler in the context of the last handler, instead of itself.

[lukewestby]

@gaearon just so i understand, could you elaborate on why a solution that requires a change to combineReducers wouldn't be allowed? i'm discussing this issue with some other folks and i want to be sure that my explanation is accurate.

[yelouafi]

@jlongster

I've been meaning to study redux-sagas so I wonder how this whole thing can fit together.

Would be great to read your thoughts on it. I know you've been experimenting with csp so I'm really interested in your opinion (and critics also) on the redux-saga model

[jlongster]

@yelouafi my initial impression is it looks awesome :) I'm very attracted to that style of API! I don't know how you represent effects, but the put/take API looks awesome. (sorry if this thread is getting derailed)

[yelouafi]

(@gaearon sorry too for this deviation) @jlongster happy you like it :). Effects are plain objects created using functions like take, put, call... the middleware 'executes' the effects & the tests just check the yielded effects are correct

[ccorcos]

While I'm very excited for this discussion, I'm sorry that I have to admit that I think this is a very terrible idea.

Before telling you why, let me embrace the concept just a little bit.

This reminds me a lot of Meteor's Tracker. I made a presentation and a simplified implementation for people to learn and understand. You could definitely use this same technique to encapsulate computations and access some larger context as you propose.

If you've ever use Meteor, and particularly, Blaze, you'll see that this leads to an amazingly magical programming experience. But the problem with tracker, specifically is in a larger applications, is it becomes very challenging to keep track of whats going on, in what order, and what the performance implications are for any line of code -- something that Redux does a great job at disentangling.

---

My main objection to this approach is that your functions are no longer pure and side-effect free. A better approach would be to use something like a Task to encapsulate an action. Now you can use the full algebraic powers of map, concat, chain, sequnce, and liftA2 to compose side-effects in the same way as you combine actions.

For example, in order to prevent action collisions in redux, you need to pass the dispatch function down to the children, and wrap the actions in a context.

items.map((item, i) => {
  const 
   = (action) => dispatch({type: 'child_action', idx: i, action})
  return <Item data={item} dispatch={childDispatch}/>
})

All that we're really doing here is mapping over a function. We can do this because a function is essentially a funtor if you think of it has holding an eventual value.

# R.map is from Ramda.js
const toChildAction = (action) => ({type: 'child_action', idx: i, action})
const childDispatch = R.map(toChildAction, dispatch)

You can do the same thing with Tasks as well! Suppose you have a task thats going to fetch some data and return an action.

const getFriendsTask = new Task((rej, res) => {
  getFriends()
    .then((friends) => res({type: 'set_friends', friends}))
    .catch((error) => res({type: 'get_friends_error', error}))
})

The parent component now needs to deal with this task, just as with actions, and wrap it up in some context so the action makes it back to the proper reducer. Well, you can map over it just the same!

const childGetFriendsTask = R.map(toChildAction, getFriendsTask)

Now, this may be a little too much for Redux -- I understand that Redux tries to be a simplification of all this functional programming power. But I think under the hood, you'll be a lot better off keeping everything pure, as it currently is, and leveraging monads and functors to do all the heavy lifting.

[lukewestby]

I don't think this proposal is exclusive of using something Task-like to represent the effect. What about the current discussion prevents "throwing" or enqueuing a Task to be run later by the imperative shell which captures it? If you've got something else in mind that leverages Tasks differently, could you provide an example that illustrates usage that we can contrast to the ideas already proposed?

[lukewestby]

It seems, by the way, that the concepts we're discussing right now are not completely dissimilar from Ember's runloop https://github.com/eoinkelly/ember-runloop-handbook

[ccorcos]

@lukewestby when you throw an error, that is a side-effect. Rather than throw/catch its best to capture the behavior of a program using a proper data structure. For example, here's what you get when you're using throw/try/catch:

function doSomething() {
  // ...
  if (someProblem) {
    throw new Error('there was some issue')
  } else {
    return result
  }
}

try {
  const result = doSomething()
  const formattedResult = formatResult(result)
  // ...
} catch(e) {
  if (error.message == 'some kind of error') {
    // display a certain message
  } else {
    // if we dont know this error, then dont eat it
    throw e
  }
}

What I (and funtional programming in general) would argue is that you're just not using the right data structure to encapsulate the program. If doSomething could possibly fail, then you need encode that in the data structure that's returned. The Either monad is a great example for this.

function doSomething() {
  // ...
  if (someProblem) {
    return Either.Left.of('there was some issue')
  } else {
    return Either.Right.of(result)
  }
}

const result = doSomething()
// map only applies to Either.Right, but Either.Left just returns itself
const formattedResult = result.map(formatResult)

if (formattedResult.isLeft) {
  // handle the error cases
} else {
  // handle the formattedResult.value
}

Notice in the second example doSomething is pure. There are no side-effects like throw which magically have effects outside of the function. Instead, you encode the fact that this function can have an error by using a proper data structure.

When I was referring to using a Task earlier, I meant it in this sort of way. Using a Task and returning a Task from a function allows the function to remain pure. Thus we can test it easier, and it makes the entire thing easier to reason about. But as soon as we start instroducing side-effects into our functions we lose all the guarantees and benefit that purity gives us...

[lukewestby]

I'm completely onboard with a solution which can leave everything to the return value as long as it isn't too awkward the way that redux-loop can be, but I have to dispute that Tasks are easier to test because they are opaque. How would I call a reducer that returns a Task and assert that the resulting Task will perform the effect I expect?

[robcolburn]

Maybe just reducers the ability to dispatch? That way they can act like normal, and apply future action? You could theoretically intercept that to control timing, or to route to discreet applications.

https://gist.github.com/robcolburn/3f537a182931f0b297ac

[ccorcos]

@lukewestby as with all side-effects, you'll have to mock out the side-effect. For example, you could mock out window.fetch to return some value and ensure that when you fork the Task, it returns the expected action.

[acjay]

@ccorcos

Rather than throw/catch its best to capture the behavior of a program using a proper data structure.

That's debatable. If you check out a link I shared earlier, the author makes some great points of the downsides of explicitly using monads, functors, and the like. I'm not entirely sold, but I think it brings up some good points.

I'm not sure that throwing effects necessarily makes a reducer impure, if the standard is that they shouldn't re-inject data into a reducer. But if they're simply resumed, that seems pure to me.

[acjay]

In all the talk about alternatives, I think one option is being left out, which is to represent effects statically as state, making effect enactors regular store subscriber/dispatchers.

So there might be a fetches key in the store, with an an array of representations of what should be fetched. Reducers for actions that should result in fetches merge their requests into this array. A fetcher subscribes to the store, monitors for new fetches, and dispatches success and error actions.

This was the discussion from #1182. It's quite simple, with no middleware, no continuations, no generators, and no impurities. @yelouafi made the point, however, that this requires bundling up action info into these effect descriptions, whereas redux-saga simply listens for the actions directly and makes direct use of their payloads.

But again, I'm still not clear on how the desire for fractal/composable architecture plays into the question of sagas vs. monads vs. continuations vs. static descriptions vs. whatever else for effects.

[jdubray]

Dan,

perhaps you would consider taking a deeper look at the SAM pattern? Let's forget about State-Action-Model for a second and just consider the Praxos protocol where state alignment is achieved with three roles: Proposers (Actions in SAM), Acceptors (Model in SAM) and Learners (State function in SAM), what you realize is that Redux, with its choice of making the actions and model anemic and concentrating all the business logic (to propose, accept and learn) in the reducer has created that situation. How could you ever think about composing reducers then?

The Redux Trilogy: Thunks and Sagas should give you a hint that indeed there is somewhat of a problem in the factoring of the reducer and it is what needs to be revisited.

I believe this suggestion is aligned with @acjay "It's quite simple, with no middleware, no continuations, no generators, and no impurities."

[slorber]

I agree with @acjay and @ccorcos on this (and consider throwing errors being impure).

What we look for here is the ability to have deeply nested reducers functions to return their value like usual, AND performing effects. We want reducers to remain pure and testable like before (including the effect part).

The writer monad was done exactly for that. Instead of returning a normal value, the reducer would then return a data structure like { result: result, effects; [{...},{...}] }, and the parent reducer will propagate automatically that structure to the top level. At the top level (middleware) you receive effect descriptors, and you are free to execute them the way you want to (a bit like what redux-saga does) @ccorcos I prefer the writer monad compared to tasks because for me it looks more testable (but both solutions are quite similar anyway)?

So yes, with this solution, you have to make all intermediate reducers aware of this writer monad infracture (including combineReducers), but is it such a big deal? Deeply nested yields look very dangerous to me and can become a huge mess. I like things being explicit, even if it involves more boilerplate. This kind of feature does not really go in the direction the community loved Redux in the first place, IMHO.

Imagine you have Reducer1 > Reducer2 > Reducer3 and reducer3 yield deep effects. Looking at Reducer1 and Reducer2, you have absolutly no idea of this effect yielding, because nothing in the code makes these effects explicit. This is bad for me :(

Yes monads are hard, yes redux-loop requires boilerplate (and does not look monadic), but at least things are explicit, and by looking at a reducer you know that you can yield effect. I don't think using a fancy syntax in order to avoid modifying combineReducers will help in the first place. This kind of feature may sound convenient right now but will likely be disliked the same way as this is now disliked IMHO.

Note that in backend architecture using serious functionnal programming, it's often the case that teams build "monad stacks". You don't need to use a single monad to solve your problem. For example, if you query a database for a record with a given ID, and use an async driver, and the network may fail (or anything else), you could have a monad stack including Promise/Either/Option. I have even seen a team abstracting their monad stack, so that they could use a simpler monad stack in test env (the idea was to be able to extract promise values in a synchronous way, ie copointed monad). And then you can also use monad transformers :) Yes it becomes quite complicated, particularly with raw Javascript where the types don't help much... But anyway this does not really apply here as the reducer case is unlikely to need such a monadic infrastructure...

---

Side note: I think the Redux-loop project aims to do too much thing. For me Redux-loop and Redux-saga are complimentary and it should not be the responsability of Redux-loop to handle ajax requests: I think its role should only be to translate reusable component actions to app-specific actions by returning effects. Then sagas can handle these app-specific actions and do the required job. I don't see the point of deciding what effect to execute directly in the state branch, and it's even worse if you need access to state from another branch while yielding the effect (as your app grows, all your effects will likely be more and more moved close to the root reducer, and state shape refactoring would become more complex...)

[kurtharriger]

It is also possible to modify composeReducers such that it is still backward compatible and doesn't require every reducer to be updated to return effects. In the branch I created here https://github.com/reactjs/redux/compare/master...kurtharriger:lift-effects reducers could return a new state withEffects or just return the new state. There are probably many other ways to do this as well.

[slorber]

@kurtharriger that's true but then the problem would become to find a convention for the data structure to return... it's easy to implement your own combineReducers yourself and no need to rely on Redux one in the first place, so it's not a big deal if Redux does not provide it directly...

---

@jlongster @sebmarkbage @gaearon I guess you don't like the this based proposal of @yelouafi. However for me the "deep yield" or "dynamic variables" solutions have exactly the same drawbacks as this: when using deep yield, the component that yields assume that it should be used in a given context. By expecting that you implicitly couple the reducer to how it is supposed to be called. For me it really looks like a new fancy this alternative.

[threepointone]

The writer monad was done exactly for that. Instead of returning a normal value, the reducer would then return a data structure like { result: result, effects; [{...},{...}] }, and the parent reducer will propagate automatically that structure to the top level. At the top level (middleware) you receive effect descriptors, and you are free to execute them the way you want to (a bit like what redux-saga does) @ccorcos I prefer the writer monad compared to tasks because for me it looks more testable (but both solutions are quite similar anyway)?

@slorber, you'd love om.next, it does exactly as you describe here. pseudo js version -

class App extends Component {
  static query = () => ql`[count]`
  onClick = () => this.props.transact({ type: 'tick' })
  render() {
    return <div onClick={this.onClick}>
      clicked { this.props.count } times
    </div>
  }
}

function read(env, key /*, params */) {
  return {
    value: env.get()[key]
  }
}

function mutate(env, action){
  if(action.type === 'tick'){
    return {
      value: {
        keys: ['count']
      },
      effect: () => env.store.swap(({ count }) =>
        ({ count: count + 1 }))
    }
  }
}

application({ read, mutate }).add(App, window.app)

(and a lot of other things).

further, it's a fully sync model(!), and all the async stuff is pushed into a send function, where you're only allowed to merge into the store. It looks kinda like this

function send({ search }, cb) {
  for(let expr of search) {
    let { key, params } = expr
    if(key === 'results') {
      searchWiki(params.query,
        (err, res) => res && cb(res))
        // you can call cb as many times as you'd like
    }
  }
}

These are interesting constraints to have, and I'm having fun exploring them.

[threepointone]

To address the OP @gaearon , this architecture is interesting in the original context -

• data fetching concerns are mostly invisible, and components receive data in a shape they ask for, despite maybe having a different shape in the store/source of truth. This relieves pressure on writing reducers/mapStateToProps.
• mutations also share the same attribute, modifying the source of truth in a specific manner, again relieving pressure on 'reducers'(ie - the mutate fn).
• further, I'm not certain what the composition store here is exactly, but I'd guess you get to compose read / mutate functions, by mapping their keys to sub keys in a combined model. This way, you can create a 'tree of apps' as it were, catching/yielding data at every level/key you need. This doesn't solve 'deep yields', but it does make it simpler to compose apps.

[tomkis]

@slorber

Side note: I think the Redux-loop project aims to do too much thing. For me Redux-loop and Redux-saga are complimentary and it should not be the responsability of Redux-loop to handle ajax requests: I think its role should only be to translate reusable component actions to app-specific actions by returning effects. Then sagas can handle these app-specific actions and do the required job. I don't see the point of deciding what effect to execute directly in the state branch, and it's even worse if you need access to state from another branch while yielding the effect (as your app grows, all your effects will likely be more and more moved close to the root reducer, and state shape refactoring would become more complex...)

I wouldn't necessarily agree. It does make sense to use saga when LLT is needed and by LLT I mean something which requires intermediate state to manage the transaction (onboarding...). However, I don't think that Sagas should be effect executors. As you said, it would make sense to declaratively yield Effects in reducers (using either redux-loop or redux-side-effects, it's just different syntax) but why would these effect executors live in Sagas? Saga is pattern for managing LLT, it has nothing to do with side effects.

Because as I have already explained, above:

Imagine you have a list of items which you may somehow sort using Drag n Drop. Your reducer is responsible for deriving the new application state, therefore the reducer is authoritative entity to define the logic. Persisting sort order is just a side effect of the fact!

I found doing these kind of things nearly impossible using redux-saga or redux-saga-rxjs, sometimes Saga simply needs to be parametrised by application state and in that case, I don't really see any benefit of using Sagas which ideally should not be aware of the application state.

[slorber]

@acjay about the link you posted here: http://blog.paralleluniverse.co/2015/08/07/scoped-continuations/

I can also understand the points of the author, but still think it can create a mess. Also the author propose a type-safe implementation by using something very similar to checked exceptions: void foo() suspends A, B so you have to declare in which context your function can be executed.

If you have ever done any Java you can notice that checked exceptions, while looking like a good idea in the first place, were a bad idea that created a lot of beginners just wrapping things in try/catch everywhere C# decided to not ship checked exceptions for these reasons btw.

And I think comparing this to exceptions (weither checked or unchecked) should just warn us. Have you ever find exceptions practical to handle complex control flow in Java or Javascript?

I don't remember anything that permits the control flow to jump on the stack having proved to be a good idea. I think generators (or redux-saga) are fine, as long as it does not jump and things remain explicit at every level.

---

@threepointone I've taken a look at Om.Next but as I'm not really fluent in Clojure yet and the doc not really good enough I think I'll take a look when it becomes released. But hope to see your experiments in plain JS about that soon :)

[slorber]

@tomkis1 sorry I don't really understand your DnD example. Why would the reducer be responsible to execute the persist effect? For me your reducer should just describe that the order has changed (maybe by emitting an business action as an effect), and the something else should handle the persistance of that new order. I think your reducer should not even have to know that the order of your items is persisted, and even less about the way it is persisted (ajax/localstorage...)