Exploring the design space of operation arguments

[jnicklas]

MiniEffection passes the task as a parameter to the generator function, but the function must not have any other arguments:

run(function*(task: Task) {
  task.spawn(...)
});

If we try to extend this to accepting additional arguments, we run into some different options on how to design this.

There are currently three ways to run an operation:

function *op(task: Task) {
  // ...
}

run(op)
run(function*(task: Task) {
  task.spawn(op)
});
run(function*() {
  yield op;
});

What happens if op takes an argument?

function *op(task: Task, value: number) {
  // ...
}

run(op, 10)
run(function*(task: Task) {
  task.spawn(op, 10)
});
run(function*() {
  yield ????????
});

The first two are solvable, but there isn't any good way of making the yield case work. We don't have a handle to the proper Task value to send in. Clearly this doesn't work at all.

What if we make op a curried function?

function op(value: number) {
  return function*(task: Task) {
    // ...
  }
}

run(op(10))
run(function*(task: Task) {
  task.spawn(op(10))
});
run(function*() {
  yield op(10)
});

This works much better! The downside is that we have to add this additional layer to any operations which takes arguments (pretty much all of them?).

I recently came up with a third option, which is interesting, but I'm not sure how I feel about it. The basic idea is that we remove the ability to yield to a generator function entirely. Instead, we have to always use task.spawn. Like this:

function op(task: Task, value: number) {
  // ...
}

run(op, 10)
run(function*(task: Task) {
  task.spawn(op, 10)
});
run(function*(task: Task) {
  yield task.spawn(op, 10)
});

The interesting thing about this is that it is actually more "honest" about the execution model. When we yield to a generator function, we do in fact spawn a subtask which runs the generator, this just makes that step explicit.

There is also something interesting about the idea that an operation is something you spawn, whereas the yield point always yields to a promise or a task, which is really a type of promise, so yield is functionally equivalent to await, except with structured concurrency magic.

I kind of both love and hate this design.

Are there any other options I haven't explored?

EDIT: a fourth, even sillier option is to combine both approaches:

function op(value: number) {
  return function*(task: Task) {
    // ...
  }
}

run(op(10))
run(function*(task: Task) {
  task.spawn(op(10))
});
run(function*(task: Task) {
  yield task.spawn(op(10))
});

• in Effection proper we call this a Context, but I'm going to consistently use Task here to simplify the terminology

[jnicklas]

This problem also calls into question the entire design of passing the task as an argument. But what are the alternatives? We already tried two other approaches to this in Effection, and both of those had their serious flaws as well.

[cowboyd]

I kind of both love and hate this design.

@jnicklas I hear you. The honest, no-magic aspect about it is appealing, however I think in order to be long-term tenable, it's an awful lot of noise to signal to require an extra task.spawn at every single yield point.

I've been thinking about this and it makes me wonder if we could perhaps resolve this underlying tension between these two syntaxes by trying to make the core of effection the simplest, most JavaScript harmonic interface that we possibly can: No magic, and honest, and then layer on generators as a coat of sugar. But using generators would be 100% completely optional, although not really done in practice.

Just a hunch, but there might be a path to unify generators, async iterators and subscriptions here. Could we make the the operation iteration externalizable as part of the public API by modeling it as an async iterator that wraps each value into what amounts to a Result<T> that could either be Interrupt | T?

run(async function(task: Task) {
  let current: unknown = undefined;
  let asyncIterator = task.spawn(op);
  while (true) {
    let promise: Promise<Interrupt|unknown> = asyncIterator.next(current);
    let next = await promise;
    if (next === Interrupt) {
      //cleanup
      return;
    } else {
      current = next; 
   }
 }
});

I think we can pretty readily translate anything into an async iterator that has an implitit Halt| unioned with the type of each step. This is all very hasty and full of unvalidated hunches, but wanted to jot it down during the weekend.

[cowboyd]

A couple of follow up thought:

1. I think my hunch about external iteration may have been off by a bit, but I still think that there may be something to idea that if we think of at its core, operations are either promises or async iterables. I think the external iteration where we force you to deal with an interrupt explicitly might be a great interface to have for building a debugger though.

2. For the case the operation-as-higher order function, I'm fine with that and it would be pretty trivial to write a task HoF that would automatically curry it for you:

export const task(function* (arg1, arg2, task) {
  //....
});

[jnicklas]

@cowboyd the more I think about this, the more I believe that the curried form is the best solution we have found. I have created two pull requests here which move us in this direction:

• 8 removes support for generator instances, instead an operation must alway be a generator function. I believe that experience in BigTest has shown that this change is justified. Also it meshes much better with possible future extensions surrounding supervision and the like.

• 9 adds a higher order function like you suggested. I decided to call this HOF operation instead of task, mostly to minimize the overloading of the term Task, since then we would have both a function called task as well as a local variable, which would obviously be confusing. I'm not entirely convinced that such a HOF is necessary, but it was at least easy to implement.