memoization of Rune execution

[harrysolovay]

Taken from #1084

Should we provide access to scope within the Rune system? (to enable the fluent API and patterns to decide how/when to cache-invalidate)

import { $accountInfo, polkadotDev } from "@capi/polkadot-dev"
import { $, createDevUsers, Scope } from "capi"

const { alexa } = await createDevUsers()

const scope = new Scope()

const accountInfo = polkadotDev.System.Account.value(alexa.publicKey)

const a = await accountInfo.run(scope)
await delay(30000)
const b = await accountInfo.run(scope) // could be different now
await delay(30000)
const c = await accountInfo.run(scope) // could be different yet again

How might one specify whether to rerun all occurrences, vs. idempotent per run, vs. per scope?

[harrysolovay]

I don't believe Scope alone is the right tool for cache management. How likely is it that users will create and pass different Scopes to run? This doesn't seem like a good flow (users sharing different Scopes between files / having to think about what Runes go in what scope. Should we invert this?

const run = new Runner() // has the cache

const accountInfo = polkadotDev.System.Account.value(alexa.publicKey)

run(accountInfo) // starts pending
run(accountInfo) // retrieves pending from cache

• Downside: users need to initialize run and share it with any file that executes a Rune
• Upside: users don't need to do so for Scopes

Then we would have cache-related methods on Runes themselves.

[tjjfvi]

@harrysolovay That doesn't really change anything other than rune.run(thingPassedAroundEverywhere) -> thingPassedAroundEverywhere.run(rune)

[tjjfvi]

Cache-invalidation is really the wrong way to think about it. If a value needs to change over time, that should be represented within Rune as multiple values yielded over time.

Taking the original example:

const scope = new Scope()

const accountInfo = polkadotDev.System.Account.value(alexa.publicKey)

const a = await accountInfo.run(scope)
await delay(30000)
const b = await accountInfo.run(scope) // could be different now
await delay(30000)
const c = await accountInfo.run(scope) // could be different yet again

this is accomplished the same way this is accomplished:

const scope = new Scope()

const accountInfo = polkadotDev.System.Account.value(alexa.publicKey)

for await (const x of accountInfo.iter(scope)) {
  console.log(x) // different each time
}

Similarly, non-idempotency is accomplished via creating unique Runes.

[harrysolovay]

That doesn't really change anything other than rune.run(thingPassedAroundEverywhere) -> thingPassedAroundEverywhere.run(rune)

The difference between the two is important: in the first <code>, there may be many thingPassedAroundEverywhere. In the second, there's only one per app. Simpler.

Cache-invalidation is really the wrong way to think about it. If a value needs to change over time, that should be represented within Rune as multiple values yielded over time.

How about long-running Runes, potentially spanning several transaction submissions+finalizations?

[tjjfvi]

The difference between the two is important: in the first <code>, there may be many thingPassedAroundEverywhere. In the second, there's only one per app. Simpler.

I don't follow -- couldn't any change we could make to how much the runner needs to be shared be made to the scope instead?

How about long-running Runes, potentially spanning several transaction submissions+finalizations?

What do you mean?

[harrysolovay]

couldn't any change we could make to how much the runner needs to be shared be made to the scope instead?

Yes. But why does one need multiple scopes? Ostensibly it would be to control what is recomputed/refetched. However, this mechanism alone doesn't provide the level of control that we likely want to offer.

How about long-running Runes, potentially spanning several transaction submissions+finalizations?

If a piece of storage is read several times over the course of a single run, but there's a chance that the storage has been mutated in between reads.

[harrysolovay]

Recap of an offline convo with @tjjfvi:

• We'll be getting rid of Scope
• All Rune executions will be memoized (global)