polkadot-js / api

Promise and RxJS APIs around Polkadot and Substrate based chains via RPC calls. It is dynamically generated based on what the Substrate runtime provides in terms of metadata.
Apache License 2.0
1.07k stars 350 forks source link

Support historical type registries. #2723

Closed jnaviask closed 4 years ago

jnaviask commented 4 years ago

Consider the following code example, given that spec contains some custom types which are required for the events call, and blockNumber is an arbitrary old block:

  const registry = new TypeRegistry();
  const api = new ApiPromise({
    provider: new WsProvider(url),
    registry,
    types: spec.types,
    typesAlias: spec.typesAlias,
  });
  await api.isReady;

  const blockHash = await api.rpc.chain.getBlockHash(blockNumber);
  const result = await api.query.system.events.at(blockHash);

This will give the following failure on Edgeware at the api.query.system.events.at call:

Unable to resolve type VoteType, it will fail on construction
Unable to resolve type VoteStage, it will fail on construction
Unable to resolve type Balance2, it will fail on construction

Why? Calling a .at() method from an old version of the chain invokes getBlockRegistry in packages/api/src/base/Init.ts, which overwrites the internal TypeRegistry when a new version is detected:

    // check for pre-existing registries
    const existingViaVersion = this.#registries.find((r) => r.specVersion.eq(version.specVersion));

    if (existingViaVersion) {
      existingViaVersion.lastBlockHash = lastBlockHash;

      return existingViaVersion;
    }

    // nothing has been found, construct new
    const registry = this._initRegistry(new TypeRegistry(), this._runtimeChain as Text, version);
    const metadata = await this._rpcCore.state.getMetadata(header.parentHash).toPromise();
    const result = { isDefault: false, lastBlockHash, metada

There is currently no way to specify custom type registries for different spec versions. How can we handle this?

A simple proposal would be to add typesSpecVersion?: Record<number, RegistryTypes>; to the RegisteredTypes interface (which ApiOptions extends), and add the specified versions to this.#registries on construction. However, we cannot fetch historical metadata until the archival connection is made, so we would have to store an incomplete VersionedRegistry. To avoid this, we could also maintain a separate object just for historical RegistryTypes and wait to populate this.#registries until the requisite connection occurs.

This implementation wouldn't fully solve the problem of supplying historical types, as we would also want to specify a historical typesAlias, but it would be a big step toward proper archival access.

jnaviask commented 4 years ago

Never mind -- I found the typesBundle feature and was able to fix this with something like this:

  const api = new ApiPromise({
    provider: new WsProvider(url),
    registry,
    types: spec.types,
    typesAlias: spec.typesAlias,
    typesBundle: {
      spec: {
        'edgeware': {
          types: [
            {
              minmax: [0, 50],
              types: spec.types || {},
            },
          ]
        }
      }
    }
  });
polkadot-js-bot commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue if you think you have a related problem or query.