[0.37] Type Generation failed `BoundedVec` (subxt macro)

[DragonDev1906]

The following code (rococo-bajun.scale was generated with a previous version) compiles with version 0.36 but fails to compile with 0.37:

#[subxt::subxt(runtime_metadata_path = "./rococo-bajun.scale")]
mod rococo_bajun {}

Error Message:

error: Type Generation failed: There are two types with the the same type path bounded_collections::bounded_vec::BoundedVec but different structure. Use `scale_typegen::utils::ensure_unique_type_paths` on your `PortableRegistry` before, to avoid this error.
 --> src/main.rs:1:1
  |
1 | #[subxt::subxt(runtime_metadata_path = "./rococo-bajun.scale")]
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: this error originates in the attribute macro `subxt::subxt` (in Nightly builds, run with -Z macro-backtrace for more info)

Command used to generate the metadata file (older version of subxt):

subxt metadata \
    --pallets System,Nft \
    --url wss://rpc-rococo.bajun.network:443 \
    > metadata/rococo-bajun.scale

[pkhry]

I've taken a look and it happens both on 0.37 and on master.

[.../scale-typegen/typegen/src/typegen/mod.rs:110:29] a = Type {
    path: Path {
        segments: [],
    },
    type_params: [],
    type_def: Primitive(
        Bool,
    ),
    docs: [],
}
[.../scale-typegen/typegen/src/typegen/mod.rs:110:29] &b = Type {
    path: Path {
        segments: [
            "staging_xcm",
            "double_encoded",
            "DoubleEncoded",
        ],
    },
    type_params: [
        TypeParameter {
            name: "T",
            ty: None,
        },
    ],
    type_def: Composite(
        TypeDefComposite {
            fields: [
                Field {
                    name: Some(
                        "encoded",
                    ),
                    ty: UntrackedSymbol {
                        id: 13,
                        marker: PhantomData<fn() -> core::any::TypeId>,
                    },
                    type_name: Some(
                        "Vec<u8>",
                    ),
                    docs: [],
                },
            ],
        },
    ),
    docs: [],
}
[.../scale-typegen/typegen/src/typegen/mod.rs:110:29] &ty.ty.path = Path {
    segments: [
        "bounded_collections",
        "bounded_vec",
        "BoundedVec",
    ],
}
[.../scale-typegen/typegen/src/typegen/mod.rs:110:29] &a.path = Path {
    segments: [],
}
[.../scale-typegen/typegen/src/typegen/mod.rs:110:29] &b.path = Path {
    segments: [
        "staging_xcm",
        "double_encoded",
        "DoubleEncoded",
    ],
}
[.../scale-typegen/typegen/src/typegen/mod.rs:110:29] path = Path {
    segments: [
        "bounded_collections",
        "bounded_vec",
        "BoundedVec",
    ],
}

So this is the debugged info about type insertion that gives the DuplicateTypePath error

// a = resolved type ID that's being inserted
// b = resolved type ID that's being stored
// ty.ty.path = path of the type that's being inserted
// a.path = actual path of type ID that's being inserted when resolved through the registry
// b.path = actual path of type ID that's being stored when resolved through the registry
// path = `Path` that's used to lookup the `ModuleIR.Types`
 dbg!(a, &b, &ty.ty.path, &a.path, &b.path, path);

Why they both bind to the boundedVec is a bit of a mystery cc @jsdw @DragonDev1906 can you generate the metadata using 0.37 version of subxt?

[jsdw]

@pkhry so to summarize, there is a DoubleEncoded type that is conflicting with a BoundedVec type? Looking at the code, everything is based around comparing paths, so I don't see how two different paths would ever conflict

[pkhry]

@pkhry so to summarize, there is a DoubleEncoded type that is conflicting with a BoundedVec type? Looking at the code, everything is based around comparing paths, so I don't see how two different paths would ever conflict

that's the issue they both have the same path in that part of the code, but when we resolve their type_ids through portable registry they will have different paths.

[DragonDev1906]

can you generate the metadata using 0.37 version of subxt?

Unfortunately I cannot, as the chain has changed since then (I created that file 5 months ago). Here are some more metadata files in case they're helpful:

• Even older file (older code on-chain), created 7 Months ago, most likely with subxt@0.15, same error: rococo-bajun-old.scale.txt
• Same chain (newer code), different endpoint URL (old one doesn't exist anymore), no error: rococo-bajun-new-35.scale.txt
• Different chain (with the bajun parachain, newer code), no error: zombie-bajun-35.scale.txt
• The closest thing I have is still somewhat recent (March), different chain (with the bajun parachain), no error: zombie-bajun-old-35.scale.txt

The only chain I've had this issue on so far was Rococo, and I cannot reproduce the issue with newly downloaded metadata files (regardless of subxt version), so something relevant has likely changed on-chain since I generated that file.

In case it helps: There have been some tests regarding async-backing on the Rococo chain, with the bajun parachain being one that used it (not sure if that helps, but that may explain why this doesn't happen on the other chains I've tested or with the current Rococo chain).

The metadata file seems to be the same, regardless of the subxt-cli version (at least where I was able to test it:

6ed083e75041401ae9d7e1b9c3eff66bc11775466fc3c953d2105cc1316ce3e4  zombie-bajun-35.scale
6ed083e75041401ae9d7e1b9c3eff66bc11775466fc3c953d2105cc1316ce3e4  zombie-bajun-36.scale
6ed083e75041401ae9d7e1b9c3eff66bc11775466fc3c953d2105cc1316ce3e4  zombie-bajun-37.scale
1f850c6726f9c764cf8a39bd3d97f5870ca15d733490ba0df8bae81889dac30e  rococo-bajun-new-35.scale
1f850c6726f9c764cf8a39bd3d97f5870ca15d733490ba0df8bae81889dac30e  rococo-bajun-new-36.scale
1f850c6726f9c764cf8a39bd3d97f5870ca15d733490ba0df8bae81889dac30e  rococo-bajun-new-37.scale

everything is based around comparing paths, so I don't see how two different paths would ever conflict

Maybe one crate has been pulled in in two different versions, that would have the same path but potentially different implementations.

EDIT: Since the chain has been changed since I technically don't need this specific file to compile anymore (I can just pull in a new one), so it isn't a blocking issue for me. It's still weird though, and likely something you want to look into regardless.

[jsdw]

Thankyou so much for all of the information! I agree that we should look into this, because I wouldn't expect old metadata files to cause an issue either, so it would be good to understand what's going on here and fix it before we encounter the issue "for real"!

[jsdw]

We had a dig, and the issue ends up being that older versions of scale-info (and thus the subxt binary) would not properly update type IDs when stripping metadata. Most of the time this is fine (because we didn't actually use the type IDs) but recent code does look at them.

We'll raise a better error about this as a result of https://github.com/paritytech/scale-typegen/pull/34, which will tell users if this is encountered and ask them to update their metadatas. But basically, I don't think there's too much to do here except use a newer version of the subxt CLI tool to generate fully correct stripped metadata :)

As such, I'll close this now, but please do re-open if I missed something!