Example:
1) User starts a "turn a photo of my bookshelf into a list of books" charm
2) User takes the photo and add that to the list of photos in the charm
3) This gets turned a list of books and rendered as charm
4) The charm also organizes the books into topics (this is a multi-step process)
5) User selects the "climate change" topic and casts a "compare view points" spell
6) A new charm that analyzes the books and shows where they agree and disagree is created
7) User buys new books, so adds a new photo to the bookshelf charm
8) This in turn updates the compare view points charm
Roughly, we have two kinds of nodes, and we need both:
1) Event handlers (/reducers?) that incorporate new information. When we see multiple concurrent writes we need to apply some kind of CRDT-like resolution. Steps 1, 2, 5, 7 are examples of this.
2) Derive nodes that compute output values dependent on input values. When we see multiple concurrent writes, the current value should come from the node working with the newest inputs and all others should be ignored. Note that if one machine is very slow, it might send writes for a computation based on old data after the faster machine already computed based on the newest inputs. We want to ignore that later write (and eventually the slower machine catches up, but that last write is idempotent). Steps 3, 4, 6 and 8 are examples.
A few complications to consider:
In the second class, some might be backed by asynchronous operations like an LLM query or querying an external database. To the pipeline this looks like type 2 nodes once it settles, but it might
take a while to settle, and we might want to process incremental updates (streaming LLMs, etc.)
thus not every machine will see all the same intermediate steps
it might not be fully deterministic
In step 5, the user references a particular entity (the "climate change") when kicking off a new process. That reference has to be stable as the data changes in step 7, so that upstream changes are correctly reflected in state 8.
Example: 1) User starts a "turn a photo of my bookshelf into a list of books" charm 2) User takes the photo and add that to the list of photos in the charm 3) This gets turned a list of books and rendered as charm 4) The charm also organizes the books into topics (this is a multi-step process) 5) User selects the "climate change" topic and casts a "compare view points" spell 6) A new charm that analyzes the books and shows where they agree and disagree is created 7) User buys new books, so adds a new photo to the bookshelf charm 8) This in turn updates the compare view points charm
Roughly, we have two kinds of nodes, and we need both: 1) Event handlers (/reducers?) that incorporate new information. When we see multiple concurrent writes we need to apply some kind of CRDT-like resolution. Steps 1, 2, 5, 7 are examples of this. 2) Derive nodes that compute output values dependent on input values. When we see multiple concurrent writes, the current value should come from the node working with the newest inputs and all others should be ignored. Note that if one machine is very slow, it might send writes for a computation based on old data after the faster machine already computed based on the newest inputs. We want to ignore that later write (and eventually the slower machine catches up, but that last write is idempotent). Steps 3, 4, 6 and 8 are examples.
A few complications to consider: