Request: Context API

[theSherwood]

Are there any plans on the roadmap for a context api? I was looking at the implementation of Solid.js' context api: https://github.com/ryansolid/solid/blob/master/packages/solid/src/signal.ts

It is minimal but it is also tangled up with the state/reactivity model. I don't know if there's any way to avoid that. I really like what Sinuous is doing with the multitude of small packages so that you only bring in what you need. It does seem like it would be a shame to add much to sinuous' super simple state/reactivity model. Is there a way to add a simple context system without messing with the reactivity model?

[luwes]

Thanks for the request @theSherwood!

Looks like a useful API but I'm gonna leave this one out of the core I think. It could be a good helper module like sinuous-context.

I'll have a look at this in the future if there are enough requests / thumbs up for this feature.

[theSherwood]

@luwes

Looks like a useful API but I'm gonna leave this one out of the core I think.

That sounds ideal, really. Context is something I almost never use. But in some few cases, it is the best thing for the job. I'll do some research and see if it within my powers to create a helper module.

[ryansolid]

I implemented Context in Solid if you want any reference, but it required the core to be aware. I use the reactive tracking hierarchy to embed it. The problem is any reactive scope can execute at any time so you have to look up the tree. DOM doesn't work well without changing renderer timing as inserts happen inside out so it isn't available on initial render. Even using the reactive graph requires some consideration as it is possible for descendants to be executed above their Provider.

There are other DI strategies, but I'd love to see if there is a simpler approach to implement. I think Context is ideal for fine-grained Reactive libraries as the Provider mechanism puts store creation back under the reactive root naturally and the way subscriptions work we have none of the performance issues that come with it for React.

[theSherwood]

I've tried a couple of things, but I'm pretty stymied as to how to make this work without it being built into the reactive model. This has turned out to be a lot trickier than I initially thought.

[ryansolid]

I should mention it doesn't absolutely need to be core to pull this off. You could intercept computation or component creation and sort of inject your own context (I do that with my mobx-jsx and ko-jsx libraries, but luckily the compiler helps there) but it is definitely more involved. I concluded core was the only practical way to go but there might be stuff I didn't think of. On the positive it can be made mostly tree shakeable minus the space it takes up on computation nodes. Although I suppose if you are coming from a no bundler sort of perspective all size counts.

[theSherwood]

@ryansolid I think this is beyond me at this point, unfortunately. Even looking at the way you did it in Solid, I can't figure how translate that to Sinuous. I can't seem to make a mapping from solid's state to sinuous' state. For example, Solid's Owner (which Solid's context relies on) doesn't map directly to Sinuous' tracking, which also seems to play much of the role of Solid's Listening. I'm guessing that tracking would need to take a context (or similar) property, but my none of my implementations have worked yet.

[luwes]

I thought some more about this concept.

Some time ago I put together a hooks implementation for Swiss here, it was adapted code from the Preact implementation https://github.com/luwes/swiss/blob/master/packages/swiss/hooks/src/create-context.js https://github.com/luwes/swiss/blob/master/packages/swiss/hooks/src/core-hooks.js#L153

I might be missing some point but I don't immediately see any real benefit for our observable/signal driven libraries. VDOM libs need some pub/sub logic to make this work for them to re-run the render function. Observables already do this out of the box I think.

Made a quick knockoff of the Solid context example, https://codesandbox.io/s/sinuous-context-fe9yg

It's not pretty but I think it kinda shows that there's not a big amount of logic needed to accomplish similar results. Please correct me if I don't get it 😄

[theSherwood]

@luwes I think the advantage that context can provide over a more straightforward reactive store is a kind of dynamic scoping. If you look at https://codesandbox.io/s/sinuous-context-gt8ye you can see that the DeeplyNested component is getting the same data as Nested despite being wrapped in another ContextProvider which should, for DeeplyNested, overshadow what is being passed down by the ContextProvider at the root of the app. Wherever components are injected into the tree, they should respect the context at that level.

[luwes]

that makes sense, I had a feeling I was missing a piece.

made some progress but the bottom-up rendering is biting me in the foot 😅 https://codesandbox.io/s/sinuous-context-88b34 (went a bit back to the React spec to make it simpler to find a solution)

I feel like there must be a better approach, also not such a big fan of adding components that don't render anything.

[theSherwood]

Okay. I think I have something working. It is currently built into my fork of sinuous' observable.js. I'm basically using a slightly modified computed to forward context down the tree. At the moment, there is a lot of duplicated code between createContext and computed that could probably be abstracted away. Alternatively, the context system could be pulled out of observable.js at the cost of duplicating most of the file's code (I think?). There may be great ways to simplify this that I don't grasp at the moment because I still don't fully understand observable.js.

The added functions:

export function getContext(key) {
  if (tracking && tracking._context) {
    if (arguments.length === 0) {
      return tracking._context;
    }
    return tracking._context[key];
  }
}

export function createContext(context, observer) {
  observer._update = updateContext;
  let value;
  let oldContext = tracking._context;

  resetUpdate(updateContext);
  updateContext();

  function updateContext() {
    const prevTracking = tracking;
    if (tracking) {
      tracking._children.push(updateContext);
    }

    const prevChildren = updateContext._children;
    _unsubscribe(updateContext);
    updateContext._fresh = true;
    // Merge contexts
    let trackingContext = tracking && tracking._context;
    oldContext = { ...oldContext, ...trackingContext };
    updateContext._context = { ...oldContext, ...context };
    tracking = updateContext;
    value = observer(value);

    // If any children computations were removed mark them as fresh.
    // Check the diff of the children list between pre and post updateContext.
    prevChildren.forEach(u => {
      if (updateContext._children.indexOf(u) === -1) {
        u._fresh = true;
      }
    });

    // If any children were marked as fresh remove them from the run lists.
    const allChildren = getChildrenDeep(updateContext._children);
    allChildren.forEach(removeFreshChildren);

    tracking = prevTracking;
    return value;
  }

  // Tiny indicator that this is an observable function.
  data.$o = true;

  function data() {
    if (updateContext._fresh) {
      updateContext._observables.forEach(o => o());
    } else {
      value = updateContext();
    }
    return value;
  }

  return data;
}

Changes to computed (it has to forward the context):

function computed(observer, value) {
  ...

  function update() {
    ...

    const prevChildren = update._children;
    _unsubscribe(update);
    update._fresh = true;
    // Forward context
    const context = tracking && tracking._context
    tracking = update;
    tracking._context = context
    value = observer(value);

    ...
  }

  ...

  function data() {
    ...
  }

  return data;
}

Changes to resetUpdate (though I'm not sure these changes make a difference?):

function resetUpdate(update) {
  // Keep track of which observables trigger updates. Needed for unsubscribe.
  update._observables = [];
  update._children = [];
  update._cleanups = [];
  update._context = {};    // This is the changed line.
}

This makes for a decent api, though it requires closures.

createContext can be used like so:

...
return html`
   <${createContext} arbitraryKeyName1=${observable} arbitraryKeyName2="Static content">
      ${() => html`
          ...
      `}
   <//>
`
...

... or like so: (Edited)

....
return createContext({key1: o(0), key2: 'static'}, () => html`...`)()
...

getContext can be used with an explicit key, like so:

...
let context = getContext('key1')
...

... or getting the entire context object like so:

...
let {key1, key2} = getContext()
...

The hierarchical shadowing effects also seem to work just fine.

I'm not sure the best way to share a demo of this, because it is tied into observable.js.

I'm suspect this could all be massively simplified.

[theSherwood]

I tried pulling bringing in a copy of the file to use only for context while using the sinuous/observable for everything else. So far, no luck. I think that with this approach, it requires the tracking system of the reactivity model used in the rendering to be the same tracking system used by the context api.

@luwes I know you are loath to add context to the core. How do you feel about 2 separate builds of sinuous? One with context, one without. I just don't see any way to do this without it being part of observable.js. But building it into observable.js seems to make it pretty simple to implement.

[luwes]

@theSherwood awesome you found a solution. I tried out your code but it returned an error. tracking was undefined.

[luwes]

I do think that extra code is too big to be added to the core, sorry, I really optimized it to be very small and since the context API is not really crucial to build an app it's better left out.

great find with adding the closure as a child of the provider, that was the missing piece in my example. https://codesandbox.io/s/sinuous-context-88b34

the difficulty for me was that the provider would get called after the consumers. Solid doesn't seem to have an issue with this.

[theSherwood]

@luwes I agree it is definitely too much to add to the core. It's more supposed to be a proof of concept. Sorry about the error. I must have made a mistake when I copied it over. I'll see if I can sort it out.

The problem with that sandbox is that any addition to the component tree doesn't respect the context scope. You can see the problem in this sandbox: https://codesandbox.io/s/sinuous-context-0lsb6

Solid doesn't seem to have an issue with this.

I think that is because in Solid, each observable keeps track of it's Owner. So all they have to do is climb the chain to find the nearest owner with the context they are looking for. Perhaps @ryansolid will correct me if I'm wrong. In contrast, Sinuous' tracking system seems a lot more ephemeral.

Edit: fixed formatting problem

[ryansolid]

Yeah that part took some work. Since I auto wrap props (including children) in functions via compilation and then map them to Object getters on the prop object a lot of the syntax friction disappears. But yes you need to wrap the children and execute inside your provider context. But it's a little bit trickier than that. If any part of downstream template returns functions it can also get executed outside of the Provider as the Provider might end up returning that function past it.

I made the provider resolve downwards in the resolveChildren function. I realized that explicitly wrapping in computations can change the execution order to happen immediately while still under the Provider. It might be simpler for Sinuous since I believe it uses the older Document Fragment version of the reconciler code and doesn't have to worry about arrays with dynamic parts since they are always bound to a node?

The Owner is convenient to handle non-tracking contexts (ie.. sampled/ignore etc..) but the Listener should still be workable outside of that. I did have to add backlinks. Every node on creation knows who the Listener is at that point so it was just a matter of storing it on the Node.

[theSherwood]

@luwes I fixed a little issue in the sample app. Is that what threw the error? It wasn't throwing an error for me. So I'll have to dig a bit to find the problem.

[ryansolid]

@theSherwood Out of curiosity is there a specific scenario where you want the Context API? It is true Angular, React, and Vue do have these hierarchical injected stores, but it isn't strictly necessary. Svelte's store approach is much more what you'd expect in a reactive library of simply just creating it and importing. Most people probably just use the module system as a way of making singletons. I sort of went out of my way to try to model Context API and because of the goals I had for Solid it was worthwhile (I used it model Suspense, and ErrorBoundaries). But is it worth a different build?

When I used KnockoutJS in my company we used a Service Locator pattern with a registry. It is a tiny bit less elegant because it's non-hierarchical, but we just registered unique keys in a registry with each store instance and used those to access the appropriate store. Basically we wrote our stores as instantiatable (not singletons) and added a layer of indirection we were able to achieve a lot of what happens here.

[theSherwood]

@ryansolid Good question. The scenario I have in mind is for the same project that requires the HMR dynamic wrapper stuff. Basically, user-generated components are going to be transcluded around the app based on tag names and other features. In software like that, I find a context api extremely helpful. It allows the component to get some data and adjust it's behavior on the basis of where it has been transcluded. Particularly if the api is more like Svelte's context api than React's context api. The component asks for context by key, rather than importing some context into the module. This makes, to my mind, for the most dynamic system possible, as the component doesn't need to know where to get the context from. The app can simply have, as a convention, a set of standard context keys that will map to certain kinds of data. A component only needs to know the right key it wants.

If you have some idea of how to do that sort of thing without a context api, I'm very interested to find out more.

I realize this is a very niche project and not a standard requirement of many apps. That said, a context api is not such an exotic feature. Seems like a feature others apart from myself would be able to derive some benefit from. So I'd like to make use of it. It seems that there are 4 possible options for that.

• Make the context additions to sinuous' observable.js super minimal, so that @luwes judges the added utility to be worth the extra code. (I'm not there yet, but I think I can get it down to 10 or 15 lines. Still up to @luwes if that cost is justified, of course.).
• Build a context api that isn't built into the core. I have no clue how to achieve this. I've attempted a couple approaches without any success.
• Sinuous is pretty flexible as to what reactive model it can use. Increase this flexibility such that calls to h can be configured to take alternate reactive models. Then I can create a souped up reactive model that includes context that simply has to conform to the interface required by sinuous.
• Fork sinuous for my project. I'd rather not go down this route, because then whatever helper modules or additions I make end up being unusable in the sinuous ecosystem, and vice versa.

[ryansolid]

Is hierarchically resolution important? Picture something like:

// store.js
const registry = new Map()
export function Store(key) {
  return registry.get(key)
}

export function registerStore(key, store) {
  if (registry.has(key)) console.error("..");
  registry.set(key, store);
}

export function unregisterStore(key) {
  registry.delete(key);
}

// somewhere else
import { registerStore } from "./store";
import CounterStore from "./CounterStore";

registerStore("counter1", new CounterStore());
// maybe somewhere else
registerStore("counter2", new CounterStore());

// in a Component file somewhere
import { Store } from "./store";

export default MyComp() {
  const counter = Store("counter2");
  return html``
}

That is more or less the pattern. I mean Context is doable here but I have to admit after @luwes comment on the wrapping children syntax I remembered why this was the starting point of where I spent less time on my non-JSX versions. With JSX I could just compile this complexity away. Solid supports Tagged Template Literals(html) and HyperScript(h) as well with in theory all the same features as the JSX version including Context, Suspense, Portals etc(except clientside hydration), but it requires the developer to follow more specific syntax that I felt was non intuitive and difficult to explain. It is worth considering it is better to find solutions that fit more naturally with the developer experience.

[theSherwood]

@ryansolid If I'm understanding your example correctly, I think Context is still important for my use case. The component shouldn't need to know which version of the data it needs, because the same component will likely be transcluded into multiple different situations in which the data it needs changes simply on the basis of the hierarchy it's in.

As a kind of high level example: the data of the app should lend itself to any number of different kinds of views, potentially meaning different kinds of navigation. The same data/components might be viewable in a spreadsheet, a list, a pinterest-style grid, a kanban board, etc. Let's say that you can change that view with a simple dropdown or hotkey and that the data/components can contain internal links. Depending on the interface/navigation scheme, you might want your internal links to behave differently. If you have Context, you can provide a callback dictating the navigation behavior of all of those internal links, simply on the basis of being in a kanban vs being in a spreadsheet.

This may not be a perfect example. And for any one example I can come up with, there will surely be other ways of handling it. But in the kinds of programs I'm talking about, these kinds of patterns come up enough that Context seems like the right way to handle it. At least, it is the best way I've seen yet.

When you have taggable units of data (that may be components) that can be displayed in multiple ways and transcluded dynamically into multiple different formats and contexts, a Context api becomes pretty useful.

[luwes]

Great points. I like the idea of having a custom reactive library with context added. That should still be possible with Sinuous, here is a doc around it though it needs to be updated with the latest changes (https://github.com/luwes/sinuous/wiki/Choose-your-own-reactive-library)

---

I made the current example work without the reactive system, demo here https://codesandbox.io/s/sinuous-context-6vz16

it would be beneficial if this could be decoupled from the reactive system but there's probably some issues with it. let me know if you find any.

---

I agree with the developer experience side, it's a shame it would require an extra closure in the root of the provider. I'm wondering if for components the children should always need to be wrapped in a closure. I think that's doable via sinuous/htm at runtime.

[theSherwood]

@luwes Fantastic! I'll dig into both options!

By the way, I have the built-in context down to about 20 lines of code. (Edit: plus comments)

it's a shame it would require an extra closure in the root of the provider. I'm wondering if for components the children would always need to be wrapped in a closure. I think that's doable via sinuous/htm at runtime.

I'm very interested to see where that goes!

[ryansolid]

@theSherwood sounds like context is exactly what you are looking for. But I had to ask the question.

@luwes I think that checks. Using the renderer as an insertion point for wrapping I hadn't considered. I was trying to think of any scenario where hoisting breaks it but from a component point of view every component below the provider will be executed either in an insert or an attribute so I think that works. I guess maybe you'd need to adapt it to handle multiple different contexts? Use some unique key generator (like a symbol) to return to correctly identify by context type.

I'd definitely be interested how you could pull off not making the developer wrap it. These sort of areas have definitely been my biggest aversion to non-compiled approaches.

[luwes]

got it working 😃, no extra closure needed for components but still need to test it properly. not sure if there are any drawbacks. released 0.24.1-beta.1 to play with it.

added support for multiple contexts to the demo too, https://codesandbox.io/s/sinuous-context-6vz16

[ryansolid]

Wait, are you using babel here? So confused. I went to look at the commit for what was changed and it looked like you detect the scenario using babel. I thought this was not compiled, so all my comments have been made from that perspective. Otherwise this is pretty trivial.

Am I looking at this right? I assumed with htm you couldn't stray from what was possible with just tagged template literals otherwise people using it non-compiled vs compiled would hit different issues. I'm just beginning to realize perhaps I've been missing something here.

EDIT: NVM.. I see there is a babel version but the runtime only version is in build.js. Hmm.. I will need to look at that since I'm unclear how you solved the inside out problem without deferring the execution.

EDIT 2: Of course, sorry was a bit slow here. You can intercept the call ahead of time with tagged template syntax (not with hyperscript) since you pass the Component in you can then just wrap the props. It does execute inside out, but it walks down the the tree first gathering the pieces. I've been grouping those approaches in my head so long I forgot the capability there. Hell my tagged template version generates almost the same output as my JSX plugin. It's only the individual attribute resolution we don't have control over, but we have complete control over creating the create call whether it be components or children etc..

...I'm done, it's the end of my work day etc....

EDIT 3: I was doubting myself for a second.. I do the exact same Tagged Template Literals version with Solid I just forgot. https://codesandbox.io/s/tagged-template-context-87ilq.

[luwes]

yup, had to update both runtime and compile steps. indeed it's just for the template literals where it would work. I have yet to try out htm's JSX to htm plugin (https://github.com/developit/htm/tree/master/packages/babel-plugin-transform-jsx-to-htm) but in theory it should translate up to JSX too.

[theSherwood]

@luwes That demo is fantastic and seems to work great. I found one small issue with it. The contextProvider can only render a single child and doesn't play nicely with dynamic components (a la #56). But that turns out to be easily remedied.

I made some modifications to reflect the kind of api I prefer for Context (more Svelte, less React), and made a test app. Everything seems to be working great as far as I can tell. Here's the link: https://codesandbox.io/s/sinuous-context-by-key-re0b5

Thanks again for all the time and effort you've put into this. I'm really liking this solution you found. Also, fixing the closure issue has made this api much nicer to work with.

[luwes]

awesome, nice demo! it's good it's possible in the Provider component but other components will not be able to do this looping through the children or at least it's not intended users would do this for each component. I'll have to see if this is a blocker for getting 0.24.0 out. did it just output one child then before?

[theSherwood]

@luwes I may not be understanding the question, but as far as I could tell, only the Provider component was having the issue (displaying only the first child). Other components seemed to work as expected.

Edit: https://codesandbox.io/s/sinuous-children-multiple-bxzrk

[luwes]

ah yes I see, I only got the first child here function Provider(props, observer) {. it should be good then. will release 0.24.0 soon 👍

[luwes]

thanks to @theSherwood the context api is available: https://github.com/theSherwood/sinuous-context

also listed it in the readme so closing here.