Change: externalize TNG hooks-context

[getify]

[Updated with suggestions from down-thread]

Proposal: External TNG Hooks-Context

Going along with some of the ideas proposed around hooks being designed as "algebraic effects",

Inspired by the idea of an "IO Monad", I'm contemplating the idea of changing TNG hooks-context to work externally to the Articulated Function (AF) rather than being saved internally. This would seemingly make hooks both more compatible with FP, but also make them more testable and debuggable.

I'm opening this issue to start sketching out ideas for what that might look like, and take feedback/debate on if that's a useful direction for TNG to go.

Here's a basic example as TNG-hooks currently works:

function foo(val = 3) {
   var [x,updateX] = useState(val);
   updateX(v => v + 1);
   useEffect(function(){
      console.log(`x: ${x + 1}`);
   });
}

foo = TNG(foo);

foo();     // x: 4
foo();     // x: 5
foo();     // x: 6

Overview

So here's what I'm now considering instead. If you call an AF directly (aka, with no context), and don't apply the effects from its resulting hooks-context, the AF itself still runs, but there's nothing observable as output.

function foo(val = 3) {
   var [x,updateX] = useState(val);
   updateX(v => v + 1);
   useEffect(function(){
      console.log(`x: ${x + 1}`);
   });
}

foo = TNG(foo);

foo();    // hooks-context object returned, but nothing printed to console
foo();    // ditto
foo();    // ditto

The effects didn't run, so that's why there were no log statements.

Each time you invoke an AF, it returns a resulting TNG hooks-context object. If you call effects() on it, its pending effects will be applied:

var context = foo();
context.effects();  // x: 4
context = foo();
context.effects();  // x: 4
context = foo();
context.effects();  // x: 4

But notice that each direct invocation of an AF without a hooks-context object passed in as the first argument will then start with its own brand new hooks-context (thus printing "x: 4" each time). :(

So, if you provide a hooks-context object as the first argument, the AF will adopt that hooks-context initially:

context = foo().effects();    // x: 4
context = foo(context).effects();    // x: 5
context = foo(context).effects();    // x: 6

Here, the second call to foo(..) included passed along the resulting context hooks-context from the previous foo() invocation, meaning it adopted that context state to start from.

Note: The effects() method returns the same hooks-context object as well, to make chaining of these method calls more ergonomic.

this === Current Hooks Context

One impact of this change will be that, to avoid intruding on the function signature (parameter list) of the original (non-Articulated) function, we're instead hijacking its this binding to "pass in" the (new) current TNG hooks-context.

Consider:

// Note: original `foo(..)` signature here doesn't have to change to 
// include the hooks-context object
function foo(x,y,z) {
   var hooksContext = this;    // `this` is the current hooks-context

   var [sum,updateSum] = useState(0);
   sum += x + y + z;
   updateSum(sum);
   console.log(`sum: ${sum}`);
}

foo = TNG(foo);

var context = foo(1,2,3);   // sum: 6

// Note: by passing a hooks-context as the first argument,
// it'ss captured by TNG, and set to the underlying `this`, instead
// of being passed in as a formal parameter.
context = foo(context,3,4,5);   // sum: 18

There are certainly trade-offs here, and it probably won't be a popular decision. But at the moment I think it's the right balance.

The benefit of the this approach is that the original foo(..) signature doesn't have to change to accommodate passing in the hooks-context. The downside is that AFs can't use their own this context. That shouldn't be too much of a limitation, though, as TNG is really designed to be used on stand-alone functions, not this-aware methods, anyway.

Return Values

We also obscure the ability to have AFs return values, since they implicitly always return a hooks-context object. We'll solve this by setting a return property on the context object which holds whatever value (if any) that is returned from that AF call.

Auto Wrapped Context

It's also a bit more inconvenient to have to pass the context at each call-site and call .effects() after, just to keep an AF stateful. So, for convenience, we can produce an automatically context-wrapped version of an AF, so that it works as it did in the original TNG design:

function foo(val = 3) {
   var [x,updateX] = useState(val);
   updateX(v => v + 1);
   useEffect(function(){
      console.log(`x: ${x + 1}`);
   });
}

foo = TNG.auto(foo);     // <-- see .auto(..)

foo();   // x: 4
foo();   // x: 5
foo();   // x: 6

This would generally be discouraged (given the downsides to testability and debuggability that motivated this whole change), but still provided for convenience and legacy reasons. And just for illustrative purposes here, auto() is basically a simple helper like:

TNG.auto = function auto(fn){
   var context;
   fn = TNG(fn);
   return function wrapped(...args){
      context = fn(context,...args).effects();
   };
}

Example: Using Hooks-Context to re-render

function hitCounter(btnID) {
   // accessing the current TNG hooks-context
   var hooksContext = this;

   var [count,updateCount] = useState(0);
   var elem = useRef();

   useEffect(function onInit(){
      elem.current = document.getElementById(btnID);
      elem.current.addEventListener("click",function onClick(){
         updateCount(x => x + 1);

         // re-render the button
         hitCounter(hooksContext,btnID).effects();
      },false);
   },[btnID]);

   useEffect(function onRender(){
      elem.value = count;
   },[count]);
}

hitCounter = TNG(hitCounter);

hitCounter("the-btn").effects();   // button initially says: "0"

// click the button, it now says: "1"
// cilck the button again, it now says: "2"
// cilck the button yet again, it now says: "3"

Note: an AF has its this bound to its own new current TNG hooks-context (not the previous context that was used to invoke the AF). In the above snippet, that value is saved as hooksContext for internal access, in this case for the re-render that the click handler does later; hooksContext will be the same object that's returned from the current invocation of the AF.

Hook Events

Also, to address the concerns of #15, we'd need a way to be "notified" in some way of all the state changes, effects, and cleanups.

We'll expose a subscribe(..) on each AF if you want to be notified of any of these. These events are fired asynchronously (on the next microtask tick).

Note: subscribe(..) is useful for a variety of tasks, from debugging, to testing, to wiring up lifecycle management for "components".

Consider:

function foo(val = 3) {
   var [x,updateX] = useState(val);
   updateX(v => v + 1);
   useEffect(function(){
      updateX(v => v + 1);
      console.log(`x is now: ${x + 2}`);
      return function(){ console.log(`cleaning things up: ${x + 2}`); };
   });
}

function onStateChange(hooksContext,hookIdx,prevValue,newValue) {
   console.log("** state:",hookIdx,prevValue,newValue);
}

function onEffect(hooksContext,effectIdx,effectFn) {
   console.log("** effect:",effectIdx);
}

function onCleanup(hooksContext,cleanupIdx,cleanupFn) {
   console.log("** cleanup:",cleanupIdx);
}

foo = TNG(foo);
foo.subscribe({ state: onStateChange, effect: onEffect, cleanup: onCleanup, });

var context = foo().effects();
// x is now: 5

context = foo(context).effects();
// cleaning things up: 5
// x is now: 7

context.reset();
// cleaning things up: 7
// ** state: 0 3 4
// ** effect: 0
// ** state: 0 4 5
// ** state: 0 5 6
// ** cleanup: 0
// ** effect: 0
// ** state: 0 6 7
// ** cleanup: 0

Note: There's an unsubscribe(..) to undo subscriptions to an AF's events.

Hooks-Context Lifecycle

Because the state of a hooks-context can be mutated asynchronously (via state updaters), especially from effects, this introduces a lot uncertainty in race conditions between one version of the state context and the next. This chaos needs to be avoided (avoidable).

A hooks-context must have a defined set of lifecycle states, with clear progression; certain operations must only be allowed for certain states.

The lifecycle of a hooks-context object is:

1. Open: A new hooks-context object is implicitly created in an Open state whenever an AF is invoked without a hooks-context. Also, calling reset() on any non-Open hooks-context object transitions it back to the Open state.

   • can be passed to an AF as its hooks-context
   • hooks can register new slots (useState(..), useEffect(..), etc)
   • state updaters can modify state slot values
   • reset() should not be called; will silently do nothing
   • effects() cannot be called; will throw an exception

2. Active: If an AF is invoked with a previously Ready hooks-context object, it transitions to the Active state and remains in that state throughout the execution of the AF.

   • cannot be passed to an AF as its hooks-context; will throw an exception
   • hooks cannot register new slots; will throw an exception
   • state updaters can modify state slot values
   • reset() can be called
   • effects() cannot be called; will throw an exception

3. Pending: A hooks-context object transitions to the Pending state at the end of an AF's execution, if any pending effects scheduled on that hooks-context.

   • cannot be passed to an AF as its hooks-context; will throw an exception
   • hooks cannot register new slots; will throw an exception
   • state updaters can modify state slot values
   • reset() can be called
   • effects() must be called to invoke pending effects and transition out of the Pending state

4. Ready: A hooks-context object transitions to Ready immediately at the end of an AF's execution, but only if no pending effects were scheduled on that hooks-context. Also, a Pending hooks-context object transitions to Ready once effects() is called, and all effects have been invoked.

   • can be passed to an AF as its hooks-context
   • hooks cannot register new slots; will throw an exception
   • state updaters can modify state slot values
   • reset() can be called
   • effects() cannot be called; will throw an exception

[JesterXL]

• Having foo become a noop and not return a value feels super impure, implying it has side effects each time you call it.
• I like the apply concept to actually execute it when you're ready.
• ... however, it leads to confusion for old people like me who say "wait, Function.apply or a different apply... or is that how the magic works?"
• is there some place in a closure you can store state? this is negated if you switch to Arrow functions and you don't need any worries of this. People see this and think it is ok to use, and in the FP world, it's not.

[getify]

Having foo become a noop and not return a value

foo(..) does return a value, it just happens to be the new/modified TNG context that it returns. I don't know of any other way to get that context "out" of the function other than to use the return value to do so.

Articulated Functions don't really "compute" things the way typical functions do. They're more like a "reducer" computing a new state, which is conceptually like the new context being returned.

... however, it leads to confusion for old people like me who

I did worry a bit about that. Since it's apply() on an object and not a function, was hoping that would help. Should it be renamed to like affect() or run() something?

People see this and think it is ok to use, and in the FP world, it's not.

I don't like the use of this here, but I'm experimenting with it as a way to transport the context into your function, as opposed to having to intrude on the parameter list to do so.

[JesterXL]

Naw, this part:

foo = TNG(foo);

foo();    // ..nothing..
foo();    // ..nothing..
foo();    // ..nothing..

That should be a ... thing... I don't know what, but if it's like an Object or Context or whatever that has your apply method, that's great.

run is cool... and a good idea. That's what it does; it runs just like a Promise does and runs and hides all the effects. You start using things like ap or then or map you start getting all the Category Theory peeps up in arms, then they start using math words at you. It's how Folktale runs tasks (cancelable Promises):

const { task } = require('folktale/concurrency/task');

const delay = (ms) => task(
  (resolver) => {
    const timerId = setTimeout(() => resolver.resolve(ms), ms);
    resolver.cleanup(() => {
      clearTimeout(timerId);
    });
  }
);

// waits 100ms
const result = await delay(100).or(delay(2000)).run().promise();
$ASSERT(result == 100);

Good luck on the this thing, man, that's... hard, heh!

[getify]

That should be a ... thing... I don't know what, but if it's like an Object or Context or whatever that has your apply method, that's great.

Sorry for my miscommunication, lemme clarify... those calls all would be returning the context object, exactly as in the next snippet.

The "nothing" there was intended to indicate that the console statements didn't run and print the "x: 4" type messages because the effects didn't run since you didn't call apply(). I'll figure out a way to make that clearer. Sorry!

[getify]

run is cool... and a good idea. That's what it does

Since what it actually does is invoke the pending effects, I'm currently thinking I'll call it effects() instead of apply() or run(). I don't want there to be a connotation that the Articulated Function hasn't run yet -- it has, its effects are what are still pending.

[JesterXL]

Oh it's not lazy? Cool, yes, effects much better then. I'd prefer a verb, but that's affect which is a canOfWorms lol

[getify]

Effects make sense to be lazy, but I can't come up with any reason why the function itself should be... generally you want new state computation (think of each Articulated Function like a reducer) to be eager and synchronous.

Effects would be where your asynchrony (like an ajax call) would be handled, and whenever that resolves that would just queue up another computation of state.

[JesterXL]

All the fp zealots yell about Promises and others being eager by default and write pages about why it's bad and I get it, but when you ask why should it be lazy, I can't answer, heh.

[getify]

Yeah. My guess is, what FP'ers mostly don't like is eager side-effects, nor do they like eager long-running tasks. That's why I'm hoping that containing things like asynchrony to lazy effects, and memoized functions, will be more "FP friendly". :)

[pkoch]

Ok, this might sound dumb, but why don't we add another attachment to the returned tngf function (like reset) that gets you buckets[tngf]?

Doesn't get you the effects affordance, but your reasoning makes it feel like a consequence, not the target of the change.

[getify]

Update: work has resumed on this after a long hiatus. And things are looking good.

I think the main code is in good shape now. I think what's left is to re-work all the tests (and add many more, for coverage sake), and docs updates. That's going to be a lot of tedious work, but it should be relatively straightforward.