index 32 not in 0 .. 31 when starting a node

[mratsim]

from @jakubgs in Discord with 1.0.0

{"lvl":"INF","ts":"2020-12-01 10:56:55.316+01:00","msg":"Fork choice initialized","topics":"beacnde","tid":22311,"file":"attestation_pool.nim:80","justified_epoch":0,"finalized_epoch":0,"finalized_root":"4d611d5b"}
libbacktrace error: no debugging symbols available. Compile with '--debugger:native'.
Traceback (most recent call last, using override)
Error: unhandled exception: index 32 not in 0 .. 31 [IndexError]

Line 80 refers to https://github.com/status-im/nimbus-eth2/blob/v1.0.0/beacon_chain/attestation_pool.nim#L80-L83

So the crash happens just as the node starts. Given the size, it's likely that it's a container using SLOTS_PER_EPOCH.

[tersec]

It's a Graffiti issue. PR coming.

[jakubgs]

How can i check what's making my grafitti so long? When I looked at the first log when starting a node it was null:

{"lvl":"INF","ts":"2020-12-01 16:19:23.573+00:00","msg":"Launching beacon node","topics":"beacnde","tid":31603,"file":"nimbus_beacon_node.nim:1307","version":"v1.0.0-fatal: not a git repository (or any of the parent directories): .git-stateofus","bls_backend":"BLST","cmdParams":[],"config":{"logLevel":"INFO","logFile":null,"eth2Network":null,"dataDir":"/root/.cache/nimbus/BeaconNode","validatorsDirFlag":null,"secretsDirFlag":null,"walletsDirFlag":null,"web3Url":"","depositContractAddress":"0x00000000219ab540356cbb839cbe05303d7705fa","depositContractDeployedAt":"11052984","nonInteractive":false,"netKeyFile":"random","netKeyInsecurePassword":false,"agentString":"nimbus","cmd":0,"bootstrapNodes":[],"bootstrapNodesFile":"","listenAddress":"0.0.0.0","tcpPort":9000,"udpPort":9000,"maxPeers":160,"nat":"any","weakSubjectivityCheckpoint":null,"finalizedCheckpointState":null,"finalizedCheckpointBlock":null,"runtimePreset":{"GENESIS_FORK_VERSION":"00000000","GENESIS_DELAY":604800,"MIN_GENESIS_ACTIVE_VALIDATOR_COUNT":16384,"MIN_GENESIS_TIME":1606824000,"ETH1_FOLLOW_DISTANCE":2048},"nodeName":"","graffiti":null,"verifyFinalization":false,"stopAtEpoch":0,"metricsEnabled":false,"metricsAddress":"127.0.0.1","metricsPort":8008,"statusBarEnabled":true,"statusBarContents":"peers: $connected_peers;finalized: $finalized_root:$finalized_epoch;head: $head_root:$head_epoch:$head_epoch_slot;time: $epoch:$epoch_slot ($slot);sync: $sync_status|ETH: $attached_validators_balance","rpcEnabled":false,"rpcPort":9190,"rpcAddress":"127.0.0.1","inProcessValidators":true,"discv5Enabled":true,"dumpEnabled":false}}

See "nodeName":"","graffiti":null.

[stefantalpalaru]

How can i check what's making my grafitti so long?

Echo the default string, somewhere in the Nim code. If you don't have Git available, what's this string going to be? https://github.com/status-im/nimbus-eth2/blob/3e4b49462bf390dba752a48f574864e9d9512e98/beacon_chain/version.nim#L18

[jakubgs]

Ah, you're right, it is long:

{
  "version":"v1.0.0-fatal: not a git repository (or any parent up to mount point /)\nStopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).-stateofus",
  "config": {
    "nodeName":"",
    "graffiti":null
  }
}

But it seems to be fine with that length of version, but the error I got was:

/build/nimbus-eth2/beacon_chain/nimbus_beacon_node.nim nimbus_beacon_node
/build/nimbus-eth2/vendor/nim-testutils/testutils/moduletests.nim main
/build/nimbus-eth2/vendor/nim-chronos/chronos/asyncmacro2.nim init
/build/nimbus-eth2/beacon_chain/spec/datatypes.nim defaultGraffitiBytes
/nix/store/08n65xsmv2vplfaxf0ziaf4ajscl7qkm-nim-unwrapped-1.2.6/nim/lib/system.nim []=
/nix/store/08n65xsmv2vplfaxf0ziaf4ajscl7qkm-nim-unwrapped-1.2.6/nim/lib/system/fatal.nim sysFatal
Error: unhandled exception: index 32 not in 0 .. 31 [IndexError]

Would be nice to have a way to avoid it calling git rev-parse --short HEAD.

[jakubgs]

Or I'd have to sed it during build time to not call it, or provide a dummy git command.

[zah]

@jakubgs, how is your build set up? Are you copying the source tree into a docker build where git is not used?

[jakubgs]

I run NixOS on my devices so I'm using Nix to build it: https://github.com/jakubgs/nixos-config/blob/master/pkgs/nimbus-eth2.nix Which means I don't have Git available at build time since Nix already provides me with all the necessary sources.

[jakubgs]

The simplest fix would be changing the buildPhase to add a git alias:

let
  rev = "83272404";
  fakeGit = pkgs.writeScriptBin "git" "echo ${rev}";
in pkgs.stdenv.mkDerivation rec {

And then provide it in PATH:

    export PATH=$PATH:${fakeGit}/bin

This way when git is called it should just echo the revision provided. But I wouldn't call that a good solution.

Some kind of compile time flag or environment variable to avoid calling git would be nice.