There are currently two types of offchain state: OffchainState.Field (a single field of state) and OffchainState.Map (a key-value map).
All offchain state is stored in a single Merkle map (Merkle tree of size 256)
There are no practical limits to the number of state fields and maps
you can use (pure) provable types of size up to ~100 field elements (~size of an action) for field and map values. (Map keys have unlimited size, since they don't need to be part of the action.)
Fields support field.get() and field.set(value) in a contract
Maps support map.get(key) and map.set(key, value)
To use offchain state, a smart contract developer must
declare an OffchainState
call offchainState.compile() and offchainState.setContractInstance() in the setup phase
add a specific onchain state field manually. This might seem brittle, but is somewhat type-safe because setContractInstance() requires the contract to have that state field.
manually add a settle() method and call it periodically to settle state
state is only available for get() after it was settled
the settle() implementation is trivial using the tools OffchainState provides
settling also involves calling createSettlementProof() outside the contract first, which is also simple from the user point of view
Some design decisions:
All info required to use offchain state is available from actions. No extra events or other external data store.
Currently, the entire merkle tree is recovered on the fly by each user, from fetched actions.
Pro: there is no extra service to set up to distribute the merkle tree
Con: This scales badly, especially with how naive the MVP implementation is
Con: The slowness of the archive node is a major bottleneck to being able to settle and use state
To me this seems like a good baseline design, to which more advanced storage and distribution mechanisms can be added later
Caveats / Future work
While writing the example, I realized that the pure set() API, which instructs offchain settlement to blindly overwrite a current value with the new one, is a major footgun and probably only useful for specific scenarios (e.g. each user of a contract is empowered to manage their own slice of state and overwrite it without taking into account old state. example: registration as a voter), but fails as a solution for the majority of use cases.
Q: Should we rename set() to overwrite() to be more honest about its limitations?
An extension that can be made easily is to add an update() method which not only takes the new state, but also the old state to replace. It only succeeds if the old state is correct, i.e. it acts exactly the same as a precondition for onchain state.
update() can be implemented such that it fails atomically on an entire account update. E.g., you make two state updates, and both fail if one of the old states is invalid.
This lets you properly implement a transfer() method, for example. The method which dispatches the transfer will only dispatch valid pairs of to/from balance changes. If one of them is outdated, both fail, so there's no way to cause an invalid balance change.
A variant of this, which reuses the implementation, would also enable map entries that can only be modified once. E.g. nullifiers. We just hard-code the "old state" hash to the initial 0 value
With update(), what we arrive at is a clean extension of the onchain state API to offchain state of arbitrary size. It completely solves the size problem. However, it doesn't fully solve the concurrency problem:
for state that is per-user, like individual account balances, concurrency is not a dealbreaker. A user has to sequence their own individual interactions - fine.
however, the current design does NOT solve the problem of shared state that is modified concurrently. A good example is the token supply in our example: The current design can only change supply (= mint or burn) one block at a time.
One idea to improve shared state handling is to introduce user-defined (or predefined) commutative actions. For example, an action could say "add this number to the state" instead of "change the state to this value".
This would solve the token supply use case example
I think this can be shipped as an extension to the current API, where commutative actions are additional inputs to the OffchainState.Field / OffchainState.Map declarations
the implementation is more involved and comes with an overhead because it means the reducer has to interact with state values, not just value hashes. it also has to disambiguate different state fields. (Efficiency-wise, both of those factors should be dominated by the Merkle hashing per action that is already done)
The final design should probably be more similar to protokit's runtime modules: A client/server like model where arbitrary computation is run in an offchain reducer to go from the previous to the next set of states, in one atomic update.
MVP design of an offchain state implementation.
update()and make the Merkle updates more efficient before releaseOffchainStateDesign
There are currently two types of offchain state:
OffchainState.Field(a single field of state) andOffchainState.Map(a key-value map).field.get()andfield.set(value)in a contractmap.get(key)andmap.set(key, value)To use offchain state, a smart contract developer must
OffchainStateoffchainState.compile()andoffchainState.setContractInstance()in the setup phasesetContractInstance()requires the contract to have that state field.settle()method and call it periodically to settle stateget()after it was settledsettle()implementation is trivial using the toolsOffchainStateprovidescreateSettlementProof()outside the contract first, which is also simple from the user point of viewSome design decisions:
Caveats / Future work
While writing the example, I realized that the pure
set()API, which instructs offchain settlement to blindly overwrite a current value with the new one, is a major footgun and probably only useful for specific scenarios (e.g. each user of a contract is empowered to manage their own slice of state and overwrite it without taking into account old state. example: registration as a voter), but fails as a solution for the majority of use cases.Q: Should we rename
set()tooverwrite()to be more honest about its limitations?An extension that can be made easily is to add an
update()method which not only takes the new state, but also the old state to replace. It only succeeds if the old state is correct, i.e. it acts exactly the same as a precondition for onchain state.update()can be implemented such that it fails atomically on an entire account update. E.g., you make two state updates, and both fail if one of the old states is invalid.transfer()method, for example. The method which dispatches the transfer will only dispatch valid pairs of to/from balance changes. If one of them is outdated, both fail, so there's no way to cause an invalid balance change.With
update(), what we arrive at is a clean extension of the onchain state API to offchain state of arbitrary size. It completely solves the size problem. However, it doesn't fully solve the concurrency problem:One idea to improve shared state handling is to introduce user-defined (or predefined) commutative actions. For example, an action could say "add this number to the state" instead of "change the state to this value".
OffchainState.Field/OffchainState.MapdeclarationsThe final design should probably be more similar to protokit's runtime modules: A client/server like model where arbitrary computation is run in an offchain reducer to go from the previous to the next set of states, in one atomic update.