polkadot-js / api

Promise and RxJS APIs around Polkadot and Substrate based chains via RPC calls. It is dynamically generated based on what the Substrate runtime provides in terms of metadata.
Apache License 2.0
1.07k stars 350 forks source link

How can we merge types-lookup generated from different metadata? #5664

Closed 0xverin closed 11 months ago

0xverin commented 1 year ago

I have two different metadata files, and I generated separate typegen outputs for each. I want to configure my tsconfig.json file like this: "@polkadot/types/lookup": ["chain1-types-lookup path", "chain2-types-lookup path"]. However, this configuration doesn't seem to work as expected. I can only access the types from chain1-types-lookup. Is it not supported to have multiple types-lookup configurations, or am I using it incorrectly?

jacogr commented 1 year ago

You can read about TS path config here -

The tsc compiler will pull the first matching path.

For declratation merging (which is what the loopup types are), you can look at the TS documentation -

Merging is only possibly when subsequent sections has exactly the same form, otherwise the TypeScript compiler will complain.

grumpygreenguy commented 1 year ago

@jacogr thanks for the answer.

One thing I was wondering is whether it'd be possible to somehow splice the metadata from both chains into a single augmentation? Either by combining the metadata inputs or some unholy manipulation of the typegen outputs? :thinking:

Alternatively, is there some recommended practice for projects that need to interface with multiple chains? (outside of liberally sprinkling our code with as and any which kind of defeats the purpose of the type system)

jacogr commented 1 year ago

You would need to augmented imports to apply the types, for instance

// augmentation 1
import './something.js';
// augmentation 2
import './somethingElse.js'

Assuming they type structures don't conflict, i.e. extra fields added, extra methods added in a pallet (for instance you can never merge Kusama & Polkadot since they do have conflicts inside pallets with regards to method params), it would augment. TypeScript only allows clean merges in the compiler - this is what we rely on, there are no manual merges, the type definitions are all handled by TypeScript, so the rules that apply there apply to augmentations.

0xverin commented 1 year ago

You would need to augmented imports to apply the types, for instance

// augmentation 1
import './something.js';
// augmentation 2
import './somethingElse.js'

Assuming they type structures don't conflict, i.e. extra fields added, extra methods added in a pallet (for instance you can never merge Kusama & Polkadot since they do have conflicts inside pallets with regards to method params), it would augment. TypeScript only allows clean merges in the compiler - this is what we rely on, there are no manual merges, the type definitions are all handled by TypeScript, so the rules that apply there apply to augmentations.

This approach seems to be ineffective for importing the types-lookup separately. I have tried importing it separately, but it doesn't work. It seems that as mentioned in my description, the configuration in tsconfig.json is required for it to be effective.

jacogr commented 1 year ago

If you need the tsconfig, then you are at the hands of how it works - tsconfig will inject the first path where it finds files.

For multiple tsconfig files, you can explore TypeScript composite projects and TypeScript references where a set of files would be tied together (with their respective aliases and augmentations) and combined into a single compilation output.

(This is how the api itself manages multiple augmentations for substrate, polkadot & kusama in a single overall project, YMMV since it is very particular as to setup and especially circular references)

There is nothing magical about type augmentations - it only uses what TypeScript provides with declaration merging, so is full in the hands of the TS compiler as to setup.

grumpygreenguy commented 1 year ago

(...) the api itself manages multiple augmentations for (...) in a single overall project, (...)

noob question --- does that mean it can interoperate between the three chains, or that an application using the api can choose at integration which one to use?

grumpygreenguy commented 1 year ago

(bumping to signal my interest in an answer to this question before the thread gets locked :) )

polkadot-js-bot commented 11 months ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue if you think you have a related problem or query.