Support hard derivation parts for ED25519 keys

[PierreBesson]

Currently, hard derivation is not supported for ED25519 keypairs. The following example fails:

Keypair.create_from_uri("//Alice//derivation", crypto_type=KeypairType.ED25519)

This error is thrown:

File "/app/.venv/lib/python3.11/site-packages/substrateinterface/base.py", line 298, in create_from_uri
     raise NotImplementedError('Derivation paths for this crypto type not supported')
 NotImplementedError: Derivation paths for this crypto type not supported

While it's not trivial to implement, for the sake of completeness of the python substrate API, it would be nice to be able to do this. Our current use case for it is : injecting derived grandpa keys on testnet nodes.

However only hard derivation can be supported as this key type has issues with soft derivation, see https://github.com/paritytech/substrate/issues/3294

Other reference link: https://github.com/w3f/hd-ed25519

[BulatSaif]

Is there are any milestone, when this will be implemented?

Polkadot binary seems to have derivation for all key types:

docker run -it --rm parity/polkadot:v0.9.40 key inspect "test test test test test test test test test test test junk//path//1" --scheme sr25519
Secret Key URI `test test test test test test test test test test test junk//path//1` is account:
  Network ID:        substrate 
 Secret seed:       0x9a8099853a5d7367be46f2e9a15f77b2062aeaac7de0c9483731edb39bd1101e
  Public key (hex):  0x349aa8b72d6b43cfb12e51009272983fc6f7605d2d16ae8cf6db537d57b0b713
  Account ID:        0x349aa8b72d6b43cfb12e51009272983fc6f7605d2d16ae8cf6db537d57b0b713
  Public key (SS58): 5DFgHoVHtvtm8L7cU6QRQ1TCuSAmgs6KAqoa7zgFY4eMWa6L
  SS58 Address:      5DFgHoVHtvtm8L7cU6QRQ1TCuSAmgs6KAqoa7zgFY4eMWa6L

docker run -it --rm parity/polkadot:v0.9.40 key inspect "test test test test test test test test test test test junk//path//1" --scheme Ed25519
Secret Key URI `test test test test test test test test test test test junk//path//1` is account:
  Network ID:        substrate 
 Secret seed:       0x48e7e181256a5dcb213f48cefea2903ab92335ca14fe4a4d0e1b8a05d4267e2b
  Public key (hex):  0x2e51ba0eb929a6ddea0e4ace308e159e6d6a59af10b650811157d26c7d3e83dd
  Account ID:        0x2e51ba0eb929a6ddea0e4ace308e159e6d6a59af10b650811157d26c7d3e83dd
  Public key (SS58): 5D7SLYZJxFGddHyB7nKsToxC3jY2kwehkGRJo4eF4NYHm2hz
  SS58 Address:      5D7SLYZJxFGddHyB7nKsToxC3jY2kwehkGRJo4eF4NYHm2hz

docker run -it --rm parity/polkadot:v0.9.40 key inspect "test test test test test test test test test test test junk//path//1" --scheme ecdsa
Secret Key URI `test test test test test test test test test test test junk//path//1` is account:
  Network ID:        substrate 
 Secret seed:       0x135a2d8ed2dbea743d79c414973ca47f8814883fb8f5dadadb8c6c90c2e9e4b8
  Public key (hex):  0x03e5af058bfeb9fff5475761a268f712b7d34c8aca96b3e40469be01dbc1cf4f83
  Account ID:        0x0bbcba001ecd397e7d52c05cbacf26661a4b2e818e452528efd872712e6e8f02
  Public key (SS58): KWDuTxXUXC9ydQ33jGNw2etnsXRj69D3nCG3SqoTJ7Myvy3Gv
  SS58 Address:      5CL6SuHEcvsjVM9qNSUW4K5f8ipUGFJ3HH18yWxzgGMNCJyh

If only the public key needed, I found a workaround for Ed25519:

from substrateinterface import SubstrateInterface
# WARNING: use local node, if mnemonic sent over the network it can be intercepted and decoded.
substrate = SubstrateInterface(url='ws://127.0.0.1:9944') # statemint-rpc
result = substrate.runtime_call("SessionKeys", "generate_session_keys", ['test test test test test test test test test test test junk//path//1'])
print(result.value)
# 0x2e51ba0eb929a6ddea0e4ace308e159e6d6a59af10b650811157d26c7d3e83dd

It is same output as polkadot binary:

docker run -it --rm parity/polkadot:v0.9.40 key inspect "test test test test test test test test test test test junk//path//1" --scheme Ed25519 | grep hex
  Public key (hex):  0x2e51ba0eb929a6ddea0e4ace308e159e6d6a59af10b650811157d26c7d3e83dd

[arjanz]

This issue was indeed a bit off the radar; I'll put it on the backlog for the upcoming release. There is some uncertainty regarding whether the above suggested hd-ed25519 RUST crate will work because we switched from ed25519-dalek bindings to ed25519-zebra bindings a while ago. When we know more we'll share the results here.

@BulatSaif sidenote, be careful using runtime_call() with your mnemonic as parameter, this will be sent over the network and could in theory be intercepted and decoded.

[BulatSaif]

@BulatSaif sidenote, be careful using runtime_call() with your mnemonic as parameter, this will be sent over the network and could in theory be intercepted and decoded.

Oh fair observation, I will update my previous comment with a warning.

[BulatSaif]

Is there any progress with this issue? The previous workaround with generate_session_keys rpc call stopped working: https://github.com/paritytech/polkadot-sdk/issues/2133