Collection API and pickCombine/pickMerge

[staltz]

I pushed to the collection branch an experimental feature where we borrow an API from Cycle Collection and improve performance under the hood.

The gist is:

Create many instances of a component using collection

function MyList(sources) {
  // ...

  // Expects sources.onion.state$ to be a Stream<Array<ItemState>>
  const instances$ = collection(Item, sources, itemState => itemState.key)
  // the above is the same as
  //const instances$ = collection(Item, sources)

pickCombine('DOM') is pick('DOM') + mix(xs.combine)

const childrenvnodes$ = instances$.compose(pickCombine('DOM'))

pickMerge('onion') is pick('onion') + mix(xs.merge)

const childrenreducer$ = instances$.compose(pickMerge('onion'))

---

Note: this is an experiment. That's why it's in a branch.

The motivation for this was to make onionify faster for large amounts of child components from an array. The main solution for this was pickCombine and the internal data structure of "instances" which collection builds. pickCombine is a fusion of pick+mix to avoid a combine, a flatten, and a map, and does all of these together and takes shortcuts to avoid recalculating things in vain. collection was sort a necessity from two perspectives: (1) easier than creating and isolating item components by hand, (2) with this performance improvement it would have become even harder to do it by hand.

Check the advanced example.

We're looking for feedback in order to find the most suitable API that feels quick to learn for the average programmer, while solving the problems below too. In other words, the challenge here is API design for developer experience. The challenge is not "how" to solve the technical problem.

Open problems/questions:

• [x] Solved thanks to a PR by @abaco. A child component may get double updates since it is onionified as well as the parent is. When the child component sends out a reducer to the onion sink, it will update its onion source state, but also the parent onion source state. We need to find a way to make the child avoid the second update since its redundant.
• [x] Update or replace the test "should work with an isolated list child with a default reducer"
• [x] How do we allow lenses on individual items, like we can do e.g. in example mixed?
• [x] collection() does a lot under the hood (creates the instances data structure, calls onionify, picks the key from each item state, creates item state lenses, onionifies each child component, etc). Do we want to have heavy configuration with many arguments, or do we want to break that down into other helper functions? And how?
• [x] How to avoid fragility to object equality issues? (read the thread)
• [x] How to keep the feature where we can pass a channel name instead of "onion"
• [x] This change expects item state to have a unique key. Do we still want to allow keyless items in a list?

Checklist before final release:

• [x] Rewrite official cycle examples
• [x] Rewrite todomvc
• [x] Rewrite matrixmultiplication
• [x] Write JSDocs for asCollection, pickCombine, pickMerge
• [x] asCollection to support state$ when it's a stream of objects
• [x] Implement CollectionSource
• [x] Rename asCollection to toCollection
• [x] Gather more feedback
• [x] Fix bug described by ntilwalli
• [x] Add tests for Cycle Unified
• [x] Check issue ntilwalli raised about isolateEach
• [x] Fix issue ntilwalli raised about RxJS support
• [x] Check issue ntilwalli raised about using toCollection without wrapper List component
• [x] Check issue atomrc raised about toCollection()
• [x] Check issue jvanbruegge raised about initial state
• [x] Update readme.md docs
• [x] Publish

[staltz]

I realized, while working on the PR Jan sent to add CollectionSource, that stateSource.asCollection(Item, sources): CollectionSource is kind of like a lens. It's not a lens usage in the typical sense of get/set, but it's a "lens" in the sense of "I want a different way of looking at this StateSource. Instead of treating it like a normal StateSource, I want it AS A CollectionSource". I toyed with the idea of expressing this API as an actual lenses use case, but didn't find any way that made sense.

@abaco @jvanbruegge do you see what I mean?

[jvanbruegge]

I see what you mean, but I don't see a way to make that a nice API to use without exposing internal stuff which we should avoid.

[staltz]

Yeah, I agree. I'm just thinking out loud here in case the idea can be evolved later.

[staltz]

I think toSinks() should live in a utils library. What it accomplishes can already be accomplished by (just a bit) lower level APIs, so that illustrates how it's a utility, not a foundational API. I also think we need to design the API a bit more carefully.

It can't return Si (public toSinks(config): Si), because Si is the sinks type of one instance, while this object is actually the sinks type of a list of instances.

Also needs to be considered how to post-process the picked streams, as is often needed e.g. with the DOM to wrap in a <ul>.

Compare

function List(sources: Sources): Sinks {
  const listSinks = sources.onion.asCollection(Item, sources)
    .toSinks({
      'onion': pickMerge,
      'DOM': pickCombine
    }) as any

  return {
    ...listSinks,
    DOM: listSinks.DOM.map((items: Array<VNode>) => ul(items)),
  }
}

With

function List(sources: Sources): Sinks {
  const itemsSource = sources.onion.asCollection(Item, sources)

  const vdom$ = itemsSource.pickCombine('DOM')
    .map((itemVNodes: Array<VNode>) => ul(itemVNodes))

  const reducer$ = itemsSource.pickMerge('onion')

  return {
    DOM: vdom$,
    onion: reducer$,
  }
}

As a shortcut util, toSinks() can mature its API building on top of pickCombine and pickMerge.

[staltz]

Oh, also I noticed I needed this, @jvanbruegge:

 export type ToSinksConfig<Si> =
   | ({'*': Transformer<Si>} & TransformerConfig<Si>)
+  | TransformerConfig<Si>
   | {'*': Transformer<Si>};

To support the use case where we don't have '*' in the config.

[jvanbruegge]

you can write it like this:

function List(sources: Sources): Sinks {
  return sources.onion.asCollection(Item, sources)
    .toSinks({
      'onion': pickMerge,
      'DOM': n => s => s.compose(pickCombine(n)).map(ul)
    }) as any;
}

the problem with putting it into a util library is, that you would have to access the private _sources field. And that field should be private as it is not relevant to the outside. And yes you are right, the type signature is not correct.

[staltz]

'DOM': n => s => s.compose(pickCombine(n)).map(ul) works, but probably scares beginners.

As for this._srcs, we shouldn't use the source channels as the sink channels because there may be cases where you have more sources than sinks, and the implementation was building sinks for those excess source channels. Instead, the utility can use the instances$ data inside e.g. dict: Map<string, Si> (that Si is what we want to map over).

[kylecordes]

How about a beginner-friendly helper function in-the-box, keeping the concise yet flexible syntax above right there at hand for non-beginner use?

[staltz]

@kylecordes The problem is that having two alternatives that are just cosmetically different isn't beginner friendly.

[abaco]

@staltz's comment about .asCollection being "kind of like a lens" made me think again about this syntax. I don't like it anymore because:

• We switched from collection to StateSource.asCollection in order to automatically pass the onionify channel name to the function. Indeed that's the only benefit. The disadvantages are:

  • .asCollection sounds like you're transforming the StateSource into something that still concerns state, while in fact you're creating a bunch of components as well.

  • The 'onion' source is also provided as an argument to .asCollection, which internally discards it. That's an anomaly.

• The CollectionSource name is misleading: in fact it's a collection of sinks. Also, sources are supposed to be input to components, but CollectionSource is always generated inside components.

I think a good model are component utilities, e.g. isolate. Please forgive me for proposing yet another syntax:

// look how similar these can be:
const itemSinks = isolate(Item)(sources);
const vdom$ = itemSinks.DOM;
const reducer$ = itemSinks.onion;

const itemSinksCollection = collection('onion', Item/*, s => s.key */)(sources);
const vdom$ = itemSinksCollection.pickCombine('DOM').map(ul);
const reducer$ = itemSinksCollection.pickMerge('onion');

[staltz]

@abaco Good points. Indeed I just felt how CollectionSource is not a normal "source" like others:

https://github.com/cyclejs/todomvc-cycle/blob/ee259aef9ceab274fb4c3f2bc7c830db7896743a/src/components/TaskList/List.js#L6

(I called the output just tasks, not tasksSources)

collection('onion', Item/*, s => s.key */)(sources);

This is an interesting API, because initially we had

collection(Item, sources, itemState => itemState.key)

I think your suggestion is more reusable because you could make

const collectionItems = collection('onion', Item, s => s.key)

And then use that function wherever, passing sources to it. However, it wouldn't be just a component utility because collectionItems wouldn't be Sources => Sinks, it would be Sources => {pickMerge,pickCombine}.

(Minor issue, I'd probably put the argument order as collection(Item, s=>s.key, 'onion') because the third one will more rarely need to be passed)

That said,asCollection method gives an explicit association between sources.onion and the collection of components. I think this is quite important for ability to trace how data flows. Maybe the biggest problem is the naming of "CollectionSource"? Could we name it something else, would that solve it?

[abaco]

it wouldn't be just a component utility

I know, but it's more similar to a component utility than to a transformation of the StateSource, or as you said "a different way of looking at this StateSource". The wording "as collection" actually suggests the lens analogy. And I think that's misleading.

(Minor issue, I'd probably put the argument order as collection(Item, s=>s.key, 'onion') because the third one will more rarely need to be passed)

That said,asCollection method gives an explicit association between sources.onion and the collection of components. I think this is quite important for ability to trace how data flows.

Sure, the channel name could be the last argument because most people would use the default 'onion', but if it's the first one it also gives an explicit association with the onion source. You would have to write collection('onion', every time, but that's not very different from the current onion.asCollection(. (It's not typesafe, though).

CollectionSource could be renamed into something like SinksCollection... not sure about that.

[staltz]

I think you're right with asCollection sounding like a lens. I thought about this and considered we could just follow the principle of least astonishment and just name it:

asCollection => toCollection CollectionSource => Collection

So toCollection makes sense to me since toFoo is usually considered a transformation that produces a Foo. Then we would also have a good convention for naming the return value of that method.

cont tasksCollection = sources.onion.toCollection(Task, sources)

[abaco]

Sounds good!

Just one more remark about return values. As a beginner it took me some time to understand and internalize that Task(sources) is not a constructor: it doesn't return a Task instance, it returns a sinks object. The closest thing to a Task instance is the function invocation Task(sources), which is not the same as the value it returns.

Transposing that to collections, I think the component utility analogy is again useful:

const Task1 = isolate(Task);                             // this is a component
const taskSinks = Task1(sources);                        // this a sinks object

const TaskCollection = sources.onion.toCollection(Task); // a collection (of components)
const taskSinksCollection = TaskCollection(sources);     // a collection of sinks

If toCollection is a "collection factory" we gain in conceptual clarity. It returns a collection of components, which once invoked with sources returns a collection of sinks. I wouldn't call "collection" the return value of sources.onion.toCollection(Task, sources), it's as inaccurate as calling "component" the return value of Task(sources).

This approach also suggests a way of dealing with the anomaly that the onion source is provided twice to toCollection, both as target and as argument. With the syntax sources.onion.toCollection(Task)(sources) the target of toCollection would be used for its identity (its name), while the argument of the resulting collection would be used for the actual content of state$. I think this would make it more consistent overall, also considering the possibility of reusing the collection sources.onion.toCollection(Task) with different sources.

[staltz]

@abaco I think I ultimately agree with you with sources.onion.toCollection(Task)(sources) as the end result, although I have a different view on the underlying argumentation for that.

the argument of the resulting collection would be used for the actual content of state$

I understand sources.onion.toCollection(Task) as "use the dynamic shape of the state under sources.onion to make a collection of tasks", and then TaskCollection(sources) as "use these sources to pass to each (isolated) Task in the collection". This is not just an interpretation of the API, it's important to define the semantics of the API. I'm my opinion, doing sources.onion.toCollection(Task)(someOtherSources) would give you a collection that follows the dynamic shape of sources.onion, not someOtherSources.onion. I think it's important that we define these semantics which could surface as real program behavior.

Just one more remark about return values. As a beginner it took me some time to understand and internalize that Task(sources) is not a constructor: it doesn't return a Task instance, it returns a sinks object.

I'm not sure if your comment is in favor or against the capitalized style. Lately I've been disliking the capitalized style. It originally appeared in Cycle.js because I noticed we never had capitalized names, so it was an opportunity for giving meaning to capitalized names. This was before TypeScript. When we started supporting TypeScript, basically all the types are capitalized, and that conflicts with our previous convention since a component function would not be a type.

So, while I'm not sure what to do about that convention, I'm nowadays more likely to reserve capitalized names just for types, because of TypeScript, but also because of beginner-unfriendlyness as you pointed out.

Were you using capitalized component names as a good thing in the following comments you made about sources.onion.toCollection(Task)(sources)?

[abaco]

"use the dynamic shape of the state under sources.onion to make a collection of tasks"

That's good semantics for me. Using the someOtherSources.onion instead of sources.onion would have made toCollection slightly more useful IMHO, and maybe less surprising for someone that hasn't read the docs and expect that supplying a different onion source have some effect. But your standpoint is simpler and more consistent. And people should read the docs :)

I'm not sure if your comment is in favor or against the capitalized style.

Neither in favor nor against. I still use capitalized names in Typescript, but you're right, I could switch, because when I have a Task component I never have any task variable. I'll think about it. Good hint!

My problem with thinking of Task as a sort of constructor was deeper than capitalization. I was used from OOP to creating/initializing components and get a hold of them. I guess that's one of the many difficulties of transitioning from object-oriented to functional. Typescript helps because it constantly reminds you of the type of every variable. That's also why I insisted on the choice of the types.

[ntilwalli]

@staltz I was trying to use onionify-collection along with cyclejs-sortable and it requires sibling-isolation but not parent child isolation. To do this the asCollection method need to allow me to pass in a custom isolation selector for the DOM source/sink to be applied when the collection item is created. I was discussing with @jvanbruegge and he mentioned it's not currently possible. It should be added.

I have an example repo which uses cyclejs-sortable, onionify-collection, and RxJS here: https://github.com/ntilwalli/cycle-onionify/tree/rxjs/examples/sortable

The file which calls asCollection and uses cyclejs-sortable is here: https://github.com/ntilwalli/cycle-onionify/blob/rxjs/examples/sortable/src/List.ts

[kylecordes]

I realize this might be too abstract and undefined... but I'm wondering whether the approach of cyclejs-sortable will be ideal, once the collection stuff is nailed down well. Wouldn't it be better if instead we hand cyclejs-sortable a bunch of separate pieces of VDOM, one for each draggable component?

[jvanbruegge]

I think it's more convenient the way it is currently. The only problem is that it registers event listeners and those dont receice events if the children are isolated from the parent. That's why you have to specify sibling isolation, so the parent gets events from the children

[ntilwalli]

I published a PR for customizing isolation scopes since I was already playing with it. Critiques welcome. https://github.com/staltz/cycle-onionify/pull/44

[staltz]

The problem @ntilwalli raised is a big deal. I'll add that as a blocker before releasing.

I understand that it's just the inability of specifying isolation semantics, but I'm not quite sure the solution is to add an option arg. I think we need to iterate more on the design, like we did in this thread and we got great progress.

Right now my thoughts are 'how to decouple child instantiation from collection management'.

This tweet really got me thinking: https://twitter.com/marick/status/875861363226275840

That before finalizing the design of an API, we should write a tutorial and stay aware of how natural or how arbitrary the argumentation sounds like.

I'm also thinking whether we can consider redesigning both sortable and onionify-collection together. For instance, are there other ways of designing sortable that would fit the current onionify-collection API? And what is the root cause behind the need for sibling isolation instead of total isolation?

[staltz]

Here's where I think there is some opportunity to improve the API: the recent suggestion @abaco made for sources.onion.toCollection(Task)(sources) decouples collection management (the 1st function call) from instantiation (the 2nd function call).

Maybe there is some way of writing isolate(sources.onion.toCollection(Task), scopes)(sources) where the scopes allows us to specify the isolation semantics for each child, but I'm not sure if this makes sense because it may look like an isolate of the collection while it's actually an isolate of each collection child. This is just a half-formed thought, but the idea is to favor composability over configuration. The options object approach is about configuring behavior of a monolithic thing, toCollection.

Put in other words, how can we provide the collection management feature (which only includes efficient management of instances) as a separate thing which can be composed with other features like isolation, etc?

[jvanbruegge]

The root cause for sortable to need no or sibling isolation is that it needs to listen to mosue events on elements defined in the children. With total isolation, the parent cannot access those mouse events which prevents dragging. I think the sortable API fits rather good with collection once you can specify isolation semantics. I also think that decoupling management from isolation would be great. Have to think about how to do this though.

[abaco]

@staltz If something as concise as isolate(sources.onion.toCollection(Task), scopes) allowed to isolate the collection (not each child as you propose), then toCollection could always do sibling isolation internally. We would then usually isolate collections, just like we usually isolate components.

how can we provide the collection management feature (which only includes efficient management of instances) as a separate thing which can be composed with other features like isolation, etc?

I've though about that a lot, but I couldn't find a solution back then. Hopefully with the recent input we can come up with something. What I was thinking about is a function to lift component utilities, to get collection utilities, something like:

const isolateCollection = liftToCollection(isolate);
isolateCollection(sources.onion.toCollection(Task), scopes)(sources) 

[staltz]

@jvanbruegge Okay, understood, sounds correct.

@abaco Good point about isolating the collection if the children are isolated with sibling isolation. However that means that onionify needs to give special treatment to 'DOM' and needs to know this channel name.

[abaco]

that means that onionify needs to give special treatment to 'DOM'

That's true 😕

When flexible isolation was still being discussed, I hoped there would be standard scopes to signal "no isolation", "sibling isolation" and "full isolation", which all channels would understand. This feature would be useful now, as toCollection could do sibling isolation by providing the same scope to all channels. (Somewhat off-topic, sorry).

[ntilwalli]

If something as concise as isolate(sources.onion.toCollection(Task), scopes) allowed to isolate the collection (not each child as you propose), then toCollection could always do sibling isolation internally. We would then usually isolate collections, just like we usually isolate components.

What does it mean to isolate a collection directly? As it currently stands, the collection is returned as an array which has to be set as the children of a containing element. DOM isolation happens on elements. How can you isolate an array of components without an element container?

Even if you could isolate the collection directly (?) and then defaulted to sibling isolation internally, that would actually causecyclejs-sortable to not work again. As cyclejs-sortable is currently designed, the events need to be visible to the component which creates the collection, not to the collection itself. Total isolation of the collection would cause the child events to not be visible to the containing component which composes the makeSortable into the DOM stream.

[ntilwalli]

I like the idea of having standard scopes to signal "no isolation", "sibling isolation" and "full isolation" but it still seems to necessitate the usage of an options parameter for configuration. An alternative to sending a scopeGenerators option (as in the PR) would be to allow an optional isolator function which has the signature of (key: any) => IsolatedComponent, but again that would require an options parameter.

The alternative would be to allow users to send isolated components into the API directly somehow but that goes against what I see as the main value of onionify-collection and Collection.gather which is to allow me to write code from plural states perspective and not a plural components perspective.

[abaco]

What does it mean to isolate a collection directly?

It means isolating the common sources once, and then the sinks of each child. If you wanted to use cyclejs-sortable you wouldn't isolate the collection.

I've been mulling it over, and I think the concern of toCollection is to isolate its items from each other, not from their parent, so it makes sense to just do sibling isolation. This can be done already by setting the default scope to '._'+key instead of '$'+key (not very elegant, I know). No need to know the DOM channel name.

This problem seems comparable to isolating the onion source when the array is a substate of the current state. We currently do that through a wrapper component. Maybe there's a better way, but I think we should deal with isolating the DOM source in the same way.

[jvanbruegge]

@abaco you dont need a wrapper any more. Since there is a collection source now, you can use sources.onion.select('myArray').asCollection()

[abaco]

@jvanbruegge Ah, there's already a better way! 😄 But then you also have to isolate the sink, right? The equivalent for the DOM channel would be to isolate it manually, which is too cumbersome for something you'll basically have to do every time.

Currently toCollection performs the default isolation on each channel - not "total" isolation, as it knows nothing about the channels and their isolation semantics. The default isolation, which for the DOM channel is total, is of course what people want most of the times. So one may argue that toCollection should stay as it is and isolate each channel the default way. Indeed, my proposal of isolating with '._'+key is in a way a violation of its concern, as it implicitly concerns the DOM channel and its isolation semantics. But my point is precisely that there should be a recognized way to do sibling isolation on a bunch of channels without knowing what they are. The present issue with toCollection demonstrate this need IMO.

I'm not trying to push for a particular approach. I just want to analyse what's at stake so we can come up with a good trade-off.

[staltz]

@ntilwalli

I like the idea of having standard scopes to signal "no isolation", "sibling isolation" and "full isolation"

👍 to this idea

@jvanbruegge

Since there is a collection source now, you can use sources.onion.select('myArray').asCollection()

👍

@abaco

This can be done already by setting the default scope to '._'+key instead of '$'+key (not very elegant, I know). No need to know the DOM channel name.

👍 too

Even with all those thumbs up given, I still think there should be a way of decoupling collection management from instantiation concerns, to favor composability over configuration. I gave a lot of thought to that dense piece of code that does everything, and I have some comments:

    this._instances$ = state$.fold((acc: Instances<Si>, nextState: Array<any> | any) => {
      const dict = acc.dict;
      if (Array.isArray(nextState)) {
        const nextInstArray = Array(nextState.length) as Array<Si & {_key: string}>;
        const nextKeys = new Set<string>();
        // add
        for (let i = 0, n = nextState.length; i < n; ++i) {
          const key = getKey(nextState[i]);
          nextKeys.add(key);
          if (dict.has(key)) {
            nextInstArray[i] = dict.get(key) as any;
          } else {
 /* 1 */    const scopes = {'*': '$' + key, [name]: instanceLens(getKey, key)};
 /* 2 */    const sinks = isolate(itemComp, scopes)(sources);
            dict.set(key, sinks);
            nextInstArray[i] = sinks;
          }
          nextInstArray[i]._key = key;
        }
        // remove
        dict.forEach((_, key) => {
          if (!nextKeys.has(key)) {
            dict.delete(key);
          }
        });
        nextKeys.clear();
        return {dict: dict, arr: nextInstArray};
      } else {
        dict.clear();
        const key = getKey(nextState);
 /* 1 */const scopes = {'*': '$' + key, [name]: identityLens};
 /* 2 */const sinks = isolate(itemComp, scopes)(sources);
        dict.set(key, sinks);
        return {dict: dict, arr: [sinks]}
      }
    }, {dict: new Map(), arr: []} as Instances<Si>);

Lines 1 do scope creation. Note that it's only important to create the onion scope. Scopes for other channels is just guesswork on what the end-developer wants. This is what creates the problem for cycle-sortable, because we don't want those scopes guessed by collection().

Lines 2 do instantiation. This will be decoupled once we change the API from toCollection(Task, sources) to toCollection(Task)(sources) (I'm still wondering what does toCollection() and what does the second function call return, and specially where to put pickCombine/pickMerge as methods).

So here's what I think we can do: two-step isolation. Inside our funky code, we isolate only on the onion channel. When passing the component instance, it should be isolated already in all other scopes. Something like:

sources.onion.toCollection(isolate(Task, scopesForAllChannelsExceptOnion))(sources)
  .pickCombine('DOM')
  // ...

The trick is that toCollection will create the scope for onion, but all other scopes are already provided, and toCollection will receive "IsolatedTask" as argument. The problem here is how to create scopesForAllChannelsExceptOnion. Ideally you could pass scopesForAllChannelsExceptOnion = 'foo' (just a string), but the way the isolate works is that double isolation will happen on the onion channel (first for the instanceLens, then for state.foo). That's the missing piece to solve, I guess.

[staltz]

Lol, I think I found a hacky way of solving it.

Replace 1 and 2 with

const scope = instanceLens(getKey, key);
const isolatedOnionSource = sources.onion.isolateSource(sources.onion, scope);
const isolatedOnionSource.isolateSource = void 0;
const isolatedOnionSource.isolateSinks = void 0;
const sinks = itemComp({...sources, [name]: isolatedOnionSource});
sinks[name] = sources.onion.isolateSink(sinks[name], scope);

So if itemComp is isolated, once it gets the isolatedOnionSource, it won't apply any additional isolation to it.

Could this approach break some other use case?

[abaco]

@staltz I can't follow you, sorry... If you pass isolate(Task, 'foo') to toCollection it will create several components where the DOM channels are isolated with 'foo', so basically they won't be isolated from each other. Am I missing something?

[staltz]

Oh, you're right. That's a mistake of mine, then. But maybe I can explore this direction a bit more.

[abaco]

@staltz We could combine your idea with @ntilwalli's (isolator function (key: any) => IsolatedComponent):

sources.onion.toCollection(key => isolate(Task, {DOM: '._' + key}))(sources)

[staltz]

As I'm drafting the implementation of this, I'm thinking how would it be to explain this in a README, or how to show a fellow developer how to use this while live coding, and I'm staying aware of how beginner friendly or unfriendly the API may be.

I find it hard to teach the API toCollection(s => s.key, key => isolate(Task, key))(sources) with a convincing argument. It would easily sound like "it is like this because it is like this." Also I'm afraid how beginners could feel dizzy trying to read that line.

What follows is a half-formed thought:

  const tasks = sources.onion
    .toCollection()
    .uniqueBy(s => s.key)
    .buildAs(key => isolate(Task, key))
    .call(sources)

  const vdom$ = tasks.pickCombine('DOM')
    .map(itemVNodes => div({style: {marginTop: '20px'}}, itemVNodes))

  const reducer$ = tasks.pickMerge('onion')

I know this is more code to type, but I'm just experimenting where readability outweighs writability. I don't like how long the first chain is, but I do like how easy toCollection . uniqueBy(state => state.key) reads, because uniqueBy is familiar from lodash, and a "collection unique by keys" sounds easier to grasp, specially in cases where s=>s.key is replaced with something less obvious like s => Math.floor(s.order).

PS: uniqueBy wouldn't need to be always called. When it's not called in the chain, then s => s.key is used as the default.

PS 2: here's a variant

  const tasks = sources.onion
    .toCollection()
    .uniqueBy(s => s.key)
    .buildAs(key => isolate(Task, key)(sources))

[staltz]

The last variant can't be built because sources can't be passed directly, they need to be first patched with the correct instanceLens onion scope.

[abaco]

Looks really good! I don't find it verbose at all.

I'll throw in one more variant for good measure:

const tasks = sources.onion
    .toCollection(Task)
    .uniqueBy(s => s.key)
    .withScopes(key => key)
    .call(sources)

withScopes would be easy to explain: "provide scopes for all channels except onion". I think a little easier than buildAs: "provide an already isolated component (NOTE: whatever you do to the onion channel will be ignored)". Also, the default case would be:

const tasks = sources.onion.toCollection(Task).call(sources)`

A separate question: I suspect you have good reasons to exclude something like:

const tasks = sources.onion.toCollection(Task, {
    getKey: s => s.key,
    scopes: key => key
})(sources)

Am I right? Why exactly don't you like that?

[staltz]

withScopes 👍 I haven't thought about that one. I'll think about it, specially because it's implicit isolate, versus the previously proposed explicit isolate.

About the options object, yes I considered it, and I don't have a strong argument against it, just a gut feeling that it looks more like configuration that it looks like composability.

I'm not sure what to do about isolate:

If it's explicit:

• Pros: familiar, easy to read and know what's happening, can pass scopes in a familiar way
• Cons: must remember to do it

If it's implicit:

• Pros: more likely that people will use the API the right way, less typing to do, and toCollection(Task) not toCollection()
• Cons: less control and vision over what's going on

PS: the call() part is annoying. We could just have a function instead, but I'm not sure if that will read well.

[staltz]

Another thing about the options object: it's not as well supported in IDEs for autocompletion as method calls are.

[staltz]

Just listing all the possible parameters and operations involved:

• getKey (has default: s=>s.key)
• item component
• sources
• isolation scopes (has default: key=>key)
• isolate

[ntilwalli]

I love the fluent api! I like the withScopes idea too but I don't under stand how that one function can translate to multiple scopes. Wouldn't it need to be .withScope(sourceName: string, scopeGenerator: (key: any) => any)? As in withScope('DOM', key => '._' + key), and it could be used multiple times if multiple sinks need custom isolation?

[staltz]

@ntilwalli a scope can be either a string (for all channels) or an object specifying the scope for each channel

.withScopes(key => key) (use key as the scope for all channels) .withScopes(key => {DOM: '._' + key, '*': key}) (the DOM channel has scope ._${key} but other channels have just key) .withScopes(key => {DOM: '.foo', HTTP: 'bar'})

[staltz]

Here's my latest proposal:

const tasks = sources.onion
    .toCollection(Task)
    .uniqueBy(s => s.key) // optional
    .isolateEach(key => {DOM: `.${key}`, '*': key}) // optional
    .build(sources)

So usually it's used as

const tasks = sources.onion.toCollection(Task).build(sources)

I'm suggesting build instead of a function call because the only way to support the function call is if toCollection returned a function that has methods uniqueBy and isolateEach attached to it, which is weird and potentially confusing (also for code analysis tools, in IDEs etc). Also less portable to functional languages like PureScript.

[abaco]

Very nice!

Just a remark on composition vs. configuration. If uniqueBy and isolateEach override some default parameters, that's effectively configuration, even though it reads like composition. We'll have true composition if:

• without calling uniqueBy there are no keys (we considered this in the past)

• without calling isolateEach only the onion channel gets isolated

Is this crazy?

[jvanbruegge]

:+1: I love the API with build()

We have to make sure, that

sources.onion
    .select(myArrayLens) // or select('myArray')
    .toCollection(Task)
    .build(sources)

works.

[staltz]

@abaco I'm warming up to that idea, because also it may seem strange that things are automatically (automagically?) isolated when you don't call anything related to isolate. Might be tedious to build the API so that these type signatures are satisfied:

uniqueBy :: Collection -> UniqueCollection uniqueBy :: IsolatedCollection -> IsolatedUniqueCollection isolateEach :: UniqueCollection -> IsolatedUniqueCollection isolateEach :: Collection -> IsolatedCollection

But maybe it would help.

[staltz]

@jvanbruegge yes that would work

[staltz]

without calling isolateEach only the onion channel gets isolated

The problem with this is that currently cycle/isolate doesn't support isolating some channels and not isolating others. (If some channels are unspecified, they get random string scope). But we can change that. It would be a (corner case) breaking change: If a channel's scope (or wildcard '*') is null, then that can be considered a "do not isolate".