babe no more? Manual for consensus keys?

[drandreaskrueger]

no more babe keys?

My scripts to create a 4-nodes-network with my own keys ceased to work.

The scripts were done with subkey-v2 and node-template-v2 but an older version.

The problem is that babe is now missing from the build-spec chainspec.

This is a more verbose than necessary, because I had to work backwards myself, to actually understand where the breaking changes had happened. At the bottom of the issue is my question.

grandpa

grandpa is still there:

node-template-2.0.0-7d7e74fb7-x86_64-linux-gnu build-spec --chain local | jq '.genesis.runtime.grandpa'
node-template-2.0.0-8b6fe66-x86_64-linux-gnu build-spec --chain local | jq '.genesis.runtime.grandpa'

BOTH result in

... Building chain spec
{
  "authorities": [
    [
      "5FA9nQDVg267DEd8m1ZypXLBnvN7SFxYwV7ndqSYGiN9TTpu",
      1
    ],
    [
      "5GoNkf6WdbxCFnPdAnYYQyCjAKPJgLNxXwPjwTh6DGg6gN3E",
      1
    ]
  ]
}

babe

an older node-template-2.0.0-7d7e74fb7 had babe entries:

node-template-2.0.0-7d7e74fb7-x86_64-linux-gnu build-spec --chain local | jq '.genesis.runtime.babe'

2020-03-02 00:29:37 Building chain spec
{
  "authorities": [
    [
      "5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY",
      1
    ],
    [
      "5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty",
      1
    ]
  ]
}

while the new node-template-2.0.0-8b6fe66-x86_64-linux-gnu has none:

node-template-2.0.0-8b6fe66-x86_64-linux-gnu build-spec --chain local | jq '.genesis.runtime.babe'

... Building chain spec
null

backgrounds

Back then, babe and grandpa Addresses were created like this:

subkey --sr25519 inspect "tissue change wrap drum deny actor winter tent salmon ahead carbon adjust" | grep "Address"
subkey --ed25519 inspect "tissue change wrap drum deny actor winter tent salmon ahead carbon adjust" | grep "Address"

SS58 Address: 5CFZ1B2CjYKx96ipWzu4KvR5MFjExLL3TemMXqD32BTbt6Tr
SS58 Address: 5HHiES5rmNMDSq3TLaPyL8J9kg3KrspDYxnTc1UYS2m7fGnc

And for the author.insertKey('{babe,grandpa}', 'seed', 'publickey') like this:

subkey --sr25519 inspect "tissue change wrap drum deny actor winter tent salmon ahead carbon adjust" | grep "Public key"
subkey --ed25519 inspect "tissue change wrap drum deny actor winter tent salmon ahead carbon adjust" | grep "Public key"

Public key (hex): 0x0845e5782e683f4cf37094e7e6fb1e881fd1806bcb03189b21e59f4d50ace02e
Public key (hex): 0xe7189ee80b9d23f96981344656606a203fcb8ae3c825d54945352366aba0f83a

---

Questions:

• So the babe consensus has been dropped in a recent v2.0.0 ?
• subkey has not changed, right?
• Which is the uptodate keys manual now, for node-template-2.0.0-8b6fe66 ?

The idea for my run-4-nodes-with-own-keys scripting had begun on this page "Generate your own keys" if I remember correctly. Back then I also found some good tutorial page explaining the keys, and the chainspec, somewhere.

Thanks a lot!

Have a great start into your week.

[JoshOrndorff]

Hi there. You've correctly observed that Babe has been removed from the node template. It has been replaced with Aura in the node template, and Aura will likely work just as well as Babe did for you.

The updated version of the tutorial you linked is here https://substrate.dev/docs/en/tutorials/start-a-private-network/customchain and shows how the aura config looks.

subkey has had some features added, but the usage you showed above still works as you expect. Aura and Babe even use the same crypto (sr25519) so the exact same command you used to generate babe keys will still work for aura keys.

PS Babe is still part of Substrate and can be re-installed in your own node if you prefer Babe (and I can help when/if you need help).

[drandreaskrueger]

Aura will likely work just as well as Babe did for you.

Now that makes it easy. Thank you very much!

[drandreaskrueger]

Global search&replace (babe,aura) made it work again already, almost.

It needed only two more changes.

As now aura and grandpa have slightly different lists:

      "aura": {
        "authorities": [
          "5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY",
          "5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty"
        ]
      },
      "grandpa": {
        "authorities": [
          [
            "5FA9nQDVg267DEd8m1ZypXLBnvN7SFxYwV7ndqSYGiN9TTpu",
            1
          ],
          [
            "5GoNkf6WdbxCFnPdAnYYQyCjAKPJgLNxXwPjwTh6DGg6gN3E",
            1
          ]

And the reason is - thanks a lot for pointing me to that manual - very understandable:

The format for the grandpa data is more complex because the grandpa protocol supports weighted votes. In this case we have given each validator a weight of 1

Secondly, I had to set bootNodes=[] in the chainspec.json and chainspec-raw.json, otherwise I got a

(The same bootnode (/ip4/127.0.0.1/tcp/30333) is registered with two different peer ids ...

(I am reading the bootnode id from the log of the first node, and used that for the other nodes)

All good, 9 Interconnected nodes need only 900 MB, cool.

Thank you very much!

[drandreaskrueger]

Thanks a lot, @JoshOrndorff and @shawntabrizi, for you perfect & quick help. Perhaps you are interested in this little exercise I tried today, when extending and testing my scripts:

number of interconnected nodes	Total memory used	memory per node in MB
10	833	83
20	1692	85
30	2579	86
40	3595	90
50	4697	94
80	8518	106
99	11015	111

Stability:
30 nodes, on a 2018 Laptop, run just fine. No error message whatsoever. And mostly (29 peers)!

However when starting larger numbers of nodes (on a too small machine), checking the log files, I encountered error messages like these:

• Handler initialization process is too long with PeerId("QmRvr...")
• Polling the network future took 35s
• Essential task failed. Shutting down service.

Especially the last one probably means that also the memory measurements become incorrect then? All this could be fixed somehow, but I don't care enough.

To start that many nodes on one machine is a rather academic extreme case anyways. It does not make sense to start more than a couple of nodes. Still, now that my automation worked again, I wanted to see how far I can push it.