Open Lohann opened 1 year ago
There were older variations in subkey too, so maybe we should document build steps for the important versions of subkey in a wiki somewhere.
I think we need a formal specification for Substrate-BIP39 that excludes any ambiguity, also include a formal definition for junction derivation for all supported algorithms (ED25519, SR25519 and ecdsa), without this there's the risk of the user lost its funds when trying to migrate its mneumonic pharse to wallets with different implementations.
I'm implemeting a Dart Client for interacting with Polkadot/Substrate, and I found this issue while implemeting Substrate-bip39 in Dart. Without a formal spec I'm not sure what is the correct implementation:
final n = BigInt.tryParse(code, radix: 10);
final Uint8List bytes;
if (n != null &&
n >= BigInt.zero &&
n < BigInt.parse('18446744073709551616')) {
// number
bytes = U64Codec.codec.encode(n);
} else {
// something else
bytes = StrCodec.codec.encode(code);
}
or this:
final n = BigInt.tryParse(code, radix: 10);
final Uint8List bytes;
if (n != null && n >= BigInt.zero) {
// number
bytes = toLittleEndianBytes(n);
} else {
// something else
bytes = StrCodec.codec.encode(code);
}
Would be so much easier if the standard considered everything as a byte string, but now it would break compatibility.
Also would be nice to enforce some limits and constraints, it would make the implementation safer and more predictable, especially for constrained devices like hardware wallets, ex:
1 - A valid uri cannot have more than 32 junctions, or all complaint devices must support up to 32 junctions.
2 - If the uri starts with 0x
it must be a raw hex encoded seed
3 - A valid URI and junction must contain only alphanumeric chars, ex: vessel ladder //$#(*
would be an invalid uri
4 - A junction cannot have more than 32 chars in length (either for numeric or string junctions).
5 - A numeric junction must be 256bit unsigned integer LE encoded, otherwise is a string junction SCALE encoded.
I think that everything that is substrate BIP-39 related must move to substrate-bip39 repo, including junction derivation logic, I had a lot of extra work trying to implement a complaint substrate-bip39 lib because the code is spread between substrate and substrate-bip39 repos. Having everything centralized in one repo also makes much easier for third-party libs do a native binding and support substrate-bip39.
Also if someone want to support a new signature scheme, like NIST Post Quantum ones, we can discuss how junction derivation should be implemented in one place.
If I don't miss something, currently the sources of knowledge about some key derivation details are:
Currently the wiki:
soft
derivation is supported only for sr25519
@burdges is there any RFC or "official" doc describing how our HDKD algorithm (not the bitcoin one since is not compatible) should behave for integer junctions with value ≥ 2^26?
cc @lisa-parity what about starting by adding these details to the wiki?
Maybe should not be part of the subkey
tool description but as a paragraph of https://docs.substrate.io/learn/accounts-addresses-keys/
I'd think the spec team should've written something by now, but maybe not.. @drskalman ?
You'll need the code in https://github.com/w3f/schnorrkel/blob/master/src/derive.rs too probably, but maybe wallets do their own hard derivations, not sure. Yes it should be speced it nobody did it.
We make several mistakes in substrate, some inherented from bitcoin and some new, but mostly impossible to fix.
We should've less logical similarity between hard and soft derivations, so they should somehow be much more separated than in bitcoin & similar.
In other words, the whole "seed phrase / phrase / phrase" thing makes sense for hard derivations. It's kinda a mess for soft derivations though. Ideally we'd deploy a "new fixed" soft derivation that works under an sr25519 key pair or DKG, but doing this is not on anyone's radar. I'm not even sure if it depends upon the DKG right now.
All the above says, there is limited value in the much of the currently deployed scheme, but like I hinted above there is great value in users being able to access their old accounts on niche signers like subkey. Imho the first question should be: Do we have adequate instructions for accessing every possible past variant under which someone maybe holds DOT/KSM? I suppose this is a question for the W3F tech ed team, but I want to stress that this is the part that matters.
Is there an existing issue?
Experiencing problems? Have you tried our Stack Exchange first?
Description of bug
The implementation of Substrate BIP39 is not consistent between polkadot-api and subkey for numerical junctions that have more than 64bits, example:
Using polkadot-api keyring:
Using subkey:
This section of the documentation says:
However it doesn't define a max bitlength for the numerical value, the Javascript implementation consider integers of any size:
While the Rust implementation only accepts
u64
:Which implementation is the right one?
Steps to reproduce
No response