State caching - app state

[2color]

Background

When interacting with the Aragon client, there are a couple of factors that make the initial load very slow. This is primarily because we fetch logs/events from a fixed point (a certain block number) in the past.

The initial load of the voting app for the Aragon Network Vote #2 takes around 2 minutes in the best case.

Currently, the reduced state used for rendering is cached but all past events are fetched from the beginning. That means that while the second load will render fast, if any changes – like a vote cast – happened between the first load and second load, it will take around 2 minutes for them to be visible to the user.

Goal

• Eliminate the need to refetch state that is unlikely to change
• Take chain reorgs into account
  • Here we can assume that after a certain threshold, a block isn't likely to change
• Provide the necessary states to render the different loading states https://github.com/aragonone/product/issues/27

Proposal

Currently

• Currently, apps have two kinds of state:
  • event state - state derived from streamed events
  • Unlikely to change and can be considered safe to cache
  • contract state - state derived by making calls to the contract.
• Both state types can come from the app's contract or an external contract, e.g. the token contract in the voting app.

Next steps

• Update aragonAPI's store interface to take a new init function for loading contract state (state fetched directly from the blockchain without event reduction). This should replicate the logic of the INITIALIZATION_TRIGGER and give a consistent interface to apps fetching contract state. app.state(reducer, events:, init: () => Promise<object> )
• Add external contract subscription mechanism to aragonAPI's store interface (potentially replace the current events param). Allow passing external contracts (address, abi), for which the aragon api will handle fetching and subscribing to events.
  • Still need to investigate what how this API should look and if it has any benefits over the current events array that can be passed into the store function.
• Separate aragonAPI's store web3.contract.events calls into two calls to getPastEvents and events
  • By separating the two calls we can guarantee we have the correct state for caching at a specific block number when all (given external contracts for which events are fetched) calls to getPastEvents have completed.
• Cache the reduced state conditionally when it's old enough to be considered safe and unlikely to reorg.

Potential challenges

Some of the changes are breaking API changes. This will require refactoring of the app scripts to adhere to the new API. This can be time consuming and introduce new bugs. Additionally the documentation should be updated.

Expected outcomes

• None of these changes should affect the initial load
• Subsequent loads of apps should happen faster

[sohkai]

Some notes to the above, for reference when writing documentation.

---

For sake of simplicity in determining a safe block reorg point, we are simply caching the "last safe block header" once when the store() function is used (usually when an application's WebWorker script loads, which is immediately after load for the Aragon client), rather than incrementally over time.

This shouldn't impact performance very much, because most users won't have a pinned tab open over a long duration (e.g. months), to the point where incremental caching would have been useful.

---

Right now, developers are expected to write their store() reducers in a way that is "semi-idempotent" on seeing the same events, over and over again. I mention "semi" here, because reduced caches often use external contract state (not included in the events), that may be changed outside of what is available in the event parameters.

In the worst case, as it is now, each time the client starts, each reducer is fed all events over its lifetime and will see data that was already cached.

In the improved caching scenario as outlined above, reducers will still need to abide by this "semi-idempotent" assumption, but they will just see a lot less duplicated history on reruns.

[sohkai]

Completed by #297, docs will be updated as @aragon/api@2 stabilizes with #330.