flume - a shared state tree

[drunkplato]

Keep the state tree management on the server side and notify the client-side state tree of any changes.

Currently, Realm does this with the core libs running in main or a background process. However, this is all in JS. Ideally, we could have the state tree management entirely in hoon, using the client to simply inject the data into render targets.

Milestone 1: hoon state management engine

We need to build out a system that mirrors something like mobx-state-tree in hoon to run on an Urbit ship. Each node in the tree would be mapped to a render target in the UI via a lightweight JS lib.

Patching and snapshotting needs to be implemented with a lib that maps to mobx.

Milestone 2: shared state management for collaborative documents

Essentially, we need to build LiveBlocks storage for Urbit.

Links

• automerge
• y.js

[drunkplato]

After thinking about this more, I feel that mobx-state-tree + y.js need to be combined conceptually and implemented in hoon. This may have to be a new vane in the system that serves as a collaborative state tree.

[drunkplato]

flume

Implements several data structures with CRDT resolution baked in.

pod

A pod is a shared state tree that resolves several ship's transactions.

It does so by implementing an immutable state tree that has actions mutate the tree with patch operations. Each data structure will have a patch-log, which is a history of all patches from each ship. Behind the scenes, these data structures, which start with pod- will resolve conflicts based on timestamps and patch merging.

+$  pod
  $:  channel=@ux
      patches=(map @t (list patches))
      state=pod-map
  ==

Data structures

pod-map

Should extend the basic map in hoon to handle the CRDT resolution.

+$  pod-map
  $:  id=@t 
      type=%map
      parent-id=@t
      parent-key=@t
      patches=(list patch)
      data=(map @t pod-noun)
  ==

pod-list

Should extend the basic list in hoon to handle the CRDT resolution.

+$  pod-list
  $:  id=@t 
      type=%list
      parent-id=@t
      parent-key=@t
      patches=(list patch)
      data=(list @)
  ==

pod-noun

+$  pod-noun
  $:  id=@t  
      type=%noun
      parent-id=@t
      parent-key=@t
      patches=(list patch)
      data=@
  ==

patch

+$  patch
  $:  type=(%add %update %delete)
  ==

Realm features

WindowManager

+$  pod-windows
  $:  id=@t  
      type=%list
      parent-id=@t
      parent-key=@t
      patches=(list patch)
      data=pod-window
  ==

+$  dimensions
  $:  x=@
      y=@
      height=@
      width=@
  ==

+$  window
  $:  id=@t
      owner=@p
      z-index=@
      type=?(%urbit %web %native)
      dimensions=dimensions
  ==

+$  pod-window
  $:  id=@t  
      type=%noun
      parent-id=@t
      parent-key=@t
      data=
  ==

Concepts

fleet

A fleet is a group of ships that can work on shared data structures

channel

Is a path in which the pod state tree exists and is resolved.

ship

A client in a pod. Can submit transactions to a pod-map, pod-list, and a pod-object