search

oasisprotocol / oasis-core

Performant and Confidentiality-Preserving Smart Contracts + Blockchains
https://oasisprotocol.org
Apache License 2.0
332 stars 109 forks source link

panic: runtime error: invalid memory address or nil pointer dereference #3032

Closed wjdfx closed 4 years ago

wjdfx commented 4 years ago
SUMMARY

When start oasis-node got an error: panic: runtime error: invalid memory address or nil pointer dereference

ISSUE TYPE
COMPONENT NAME

oasis-node

OASIS NODE VERSION
oasis-node --version
Software version: 20.8.1
Runtime protocol version: 0.14.0
Consensus protocol version: 0.25.0
Committee protocol version: 0.9.0
Tendermint core version: 0.33.4
ABCI library version: 0.16.2
Go toolchain version: 1.14.4
OS / ENVIRONMENT

Ubuntu 18.04

STEPS TO REPRODUCE
Log
`{"caller":"common.go:211","cur":1024,"level":"warn","max":1048576,"module":"oasis-node","msg":"the node user has a very low RLIMIT_NOFILE, consider increasing","ts":"2020-06-18T16:35:54.934234044Z"}
{"Version":"20.8.1","caller":"node.go:572","level":"info","module":"oasis-node","msg":"Starting oasis-node","ts":"2020-06-18T16:35:54.935913744Z"}
{"caller":"helpers.go:40","level":"info","module":"common/persistent","msg":"All 0 tables opened in 0s","ts":"2020-06-18T16:35:54.959449486Z"}
{"caller":"node.go:625","consensus_pk":"YMht021ec8eMkSk9vkKs+LV8+WL/8XIgttcQIKYiHlo=","level":"info","module":"oasis-node","msg":"loaded/generated node identity","node_pk":"CeK2kK5crFPeQaYmRG/y3H7xt+j+yKDFoIluRKRGXgM=","p2p_pk":"/vKW4yscxI0F8ifZTXOzfmnERS14YJc50fb7RNnlUj4=","tls_pk":"fVYqMZkZudGLtJj6j4cUrtryM6Q4jaj+HzvG3GAXAYg=","ts":"2020-06-18T16:35:54.96502765Z"}
{"caller":"helpers.go:40","level":"info","module":"mkvs/db/badger","msg":"All 0 tables opened in 0s","ts":"2020-06-18T16:35:55.080871795Z"}
{"caller":"client.go:99","level":"warn","module":"ias/proxyclient","msg":"IAS proxy is not configured, all reports will be mocked","ts":"2020-06-18T16:35:55.284963546Z"}
{"caller":"registry.go:421","err":"consensus: no committed blocks","level":"error","module":"registry/tendermint","msg":"runtime notifier: unable to get a list of runtimes","ts":"2020-06-18T16:35:55.291671601Z"}
{"caller":"node.go:770","level":"info","module":"oasis-node","msg":"starting Oasis node","ts":"2020-06-18T16:35:55.299110012Z"}
{"caller":"registry.go:360","level":"info","module":"runtime/registry","msg":"no supported runtimes configured","ts":"2020-06-18T16:35:55.302557726Z"}
{"caller":"worker.go:171","level":"info","module":"worker/storage","msg":"not starting storage worker as it is disabled","ts":"2020-06-18T16:35:55.33340542Z"}
{"caller":"worker.go:43","level":"info","module":"worker/executor","msg":"not starting executor worker as it is disabled","ts":"2020-06-18T16:35:55.333486546Z"}
{"caller":"worker.go:44","level":"info","module":"worker/txnscheduler","msg":"not starting transaction scheduler as it is disabled","ts":"2020-06-18T16:35:55.333519135Z"}
{"caller":"worker.go:41","level":"info","module":"worker/merge","msg":"not starting merge worker as it is disabled","ts":"2020-06-18T16:35:55.333548655Z"}
{"caller":"worker.go:52","level":"info","module":"worker/common","msg":"not starting common worker as it is disabled","ts":"2020-06-18T16:35:55.333572118Z"}
{"caller":"worker.go:88","level":"info","module":"worker/keymanager","msg":"not starting key manager worker as it is disabled","ts":"2020-06-18T16:35:55.333588186Z"}
{"caller":"worker.go:953","level":"info","module":"worker/registration","msg":"starting node registration service","ts":"2020-06-18T16:35:55.333600015Z"}
{"caller":"worker.go:60","level":"info","module":"worker/sentry","msg":"not starting sentry worker as it is disabled","ts":"2020-06-18T16:35:55.333625946Z"}
{"caller":"service.go:139","impl":"multiAppConn","level":"info","module":"tendermint:proxy","msg":"Starting multiAppConn service","ts":"2020-06-18T16:35:55.358251707Z"}
{"caller":"service.go:139","connection":"query","impl":"localClient","level":"info","module":"tendermint:proxy/abci-client","msg":"Starting localClient service","ts":"2020-06-18T16:35:55.358317226Z"}
{"caller":"service.go:139","connection":"mempool","impl":"localClient","level":"info","module":"tendermint:proxy/abci-client","msg":"Starting localClient service","ts":"2020-06-18T16:35:55.358351043Z"}
{"caller":"service.go:139","connection":"consensus","impl":"localClient","level":"info","module":"tendermint:proxy/abci-client","msg":"Starting localClient service","ts":"2020-06-18T16:35:55.358376246Z"}
{"caller":"service.go:139","impl":"EventBus","level":"info","module":"tendermint:events","msg":"Starting EventBus service","ts":"2020-06-18T16:35:55.358395681Z"}
{"caller":"service.go:139","impl":"PubSub","level":"info","module":"tendermint:events/pubsub","msg":"Starting PubSub service","ts":"2020-06-18T16:35:55.35841643Z"}
{"caller":"service.go:139","impl":"IndexerService","level":"info","module":"tendermint:txindex","msg":"Starting IndexerService service","ts":"2020-06-18T16:35:55.358447053Z"}
{"caller":"replay.go:260","hash":"","height":0,"level":"info","module":"tendermint:consensus","msg":"ABCI Handshake App Info","protocol-version":1638400,"software-version":"","ts":"2020-06-18T16:35:55.358555116Z"}
{"appHeight":0,"caller":"replay.go:299","level":"info","module":"tendermint:consensus","msg":"ABCI Replay Blocks","stateHeight":0,"storeHeight":0,"ts":"2020-06-18T16:35:55.360609528Z"}
{"caller":"genesis.go:55","common_pool":"8488535000000000000","last_block_fees":"0","level":"warn","mode":"init chain","module":"consensus/tendermint/abci","msg":"InitChain: moving last block fees into common pool","ts":"2020-06-18T16:35:55.366229328Z"}
{"appHash":"","appHeight":0,"caller":"replay.go:279","level":"info","module":"tendermint:consensus","msg":"Completed ABCI Handshake - Tendermint and App are synced","ts":"2020-06-18T16:35:55.426520606Z"}
{"addr":"08C0CB207D8F5F0E4830633B850C80C3F9CA5D3C","caller":"node.go:310","level":"info","module":"tendermint:consensus","msg":"This node is not a validator","pubKey":"PubKeyEd25519{60C86DD36D5E73C78C91293DBE42ACF8B57CF962FFF17220B6D71020A6221E5A}","ts":"2020-06-18T16:35:55.427804264Z"}
{"ID":"3b67db4bba2509cd6abc0d35f6f8bf4f6366eeca","caller":"node.go:509","file":"/mnt/serverdir/node/tendermint/config/node_key.json","level":"info","module":"tendermint:p2p","msg":"P2P Node ID","ts":"2020-06-18T16:35:55.452987178Z"}
{"addrs":[],"caller":"switch.go:558","level":"info","module":"tendermint:p2p","msg":"Adding persistent peers","ts":"2020-06-18T16:35:55.453109248Z"}
{"caller":"switch.go:576","ids":[],"level":"info","module":"tendermint:p2p","msg":"Adding unconditional peer ids","ts":"2020-06-18T16:35:55.4531487Z"}
{"addr":"3b67db4bba2509cd6abc0d35f6f8bf4f6366eeca@95.216.211.124:26656","book":"/mnt/serverdir/node/tendermint/config/addrbook.json","caller":"addrbook.go:175","level":"info","module":"tendermint:p2p","msg":"Add our address to book","ts":"2020-06-18T16:35:55.453314563Z"}
{"addr":"3b67db4bba2509cd6abc0d35f6f8bf4f6366eeca@0.0.0.0:26656","book":"/mnt/serverdir/node/tendermint/config/addrbook.json","caller":"addrbook.go:175","level":"info","module":"tendermint:p2p","msg":"Add our address to book","ts":"2020-06-18T16:35:55.453368381Z"}
{"caller":"service.go:139","impl":"P2P Switch","level":"info","module":"tendermint:p2p","msg":"Starting P2P Switch service","ts":"2020-06-18T16:35:55.453525371Z"}
{"caller":"service.go:139","impl":"Evidence","level":"info","module":"tendermint:evidence","msg":"Starting Evidence service","ts":"2020-06-18T16:35:55.453563847Z"}
{"caller":"service.go:139","impl":"PEX","level":"info","module":"tendermint:pex","msg":"Starting PEX service","ts":"2020-06-18T16:35:55.453620538Z"}
{"book":"/mnt/serverdir/node/tendermint/config/addrbook.json","caller":"service.go:139","impl":"AddrBook","level":"info","module":"tendermint:p2p","msg":"Starting AddrBook service","ts":"2020-06-18T16:35:55.453661304Z"}
{"caller":"service.go:139","impl":"Mempool","level":"info","module":"tendermint:mempool","msg":"Starting Mempool service","ts":"2020-06-18T16:35:55.453740393Z"}
{"caller":"service.go:139","impl":"BlockchainReactor","level":"info","module":"tendermint:blockchain","msg":"Starting BlockchainReactor service","ts":"2020-06-18T16:35:55.453769415Z"}
{"caller":"service.go:139","impl":"BlockPool","level":"info","module":"tendermint:blockchain","msg":"Starting BlockPool service","ts":"2020-06-18T16:35:55.45380464Z"}
{"caller":"service.go:139","impl":"ConsensusReactor","level":"info","module":"tendermint:consensus","msg":"Starting Consensus service","ts":"2020-06-18T16:35:55.453864177Z"}
{"caller":"reactor.go:73","fastSync":true,"level":"info","module":"tendermint:consensus","msg":"Reactor ","ts":"2020-06-18T16:35:55.45390395Z"}
{"book":"/mnt/serverdir/node/tendermint/config/addrbook.json","caller":"file.go:22","level":"info","module":"tendermint:p2p","msg":"Saving AddrBook to file","size":0,"ts":"2020-06-18T16:35:55.453947427Z"}
{"caller":"pex_reactor.go:450","level":"info","module":"tendermint:pex","msg":"Ensure peers","numDialing":0,"numInPeers":0,"numOutPeers":0,"numToDial":20,"ts":"2020-06-18T16:35:55.454032578Z"}
{"caller":"pex_reactor.go:524","level":"info","module":"tendermint:pex","msg":"No addresses to dial. Falling back to seeds","ts":"2020-06-18T16:35:55.455990609Z"}
{"address":"9fbca3364885ebc82bf5be8156bb4df0a563bbf6@34.86.145.181:26656","caller":"switch.go:693","level":"info","module":"tendermint:p2p","msg":"Dialing peer","ts":"2020-06-18T16:35:55.456063839Z"}
{"caller":"grpc.go:334","level":"info","module":"grpc/internal","msg":"starting gRPC server","ts":"2020-06-18T16:35:55.457257786Z"}
{"address":"/mnt/serverdir/node/internal.sock","caller":"grpc.go:349","level":"info","module":"grpc/internal","msg":"gRPC server started","network":"unix","ts":"2020-06-18T16:35:55.457403761Z"}
{"caller":"node.go:846","level":"info","module":"oasis-node","msg":"initialization complete: ready to serve","ts":"2020-06-18T16:35:55.457543865Z"}
{"caller":"service.go:139","impl":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","level":"info","module":"tendermint:p2p","msg":"Starting Peer service","peer":"9fbca3364885ebc82bf5be8156bb4df0a563bbf6@34.86.145.181:26656","ts":"2020-06-18T16:35:55.801698095Z"}
{"caller":"service.go:139","impl":"MConn{34.86.145.181:26656}","level":"info","module":"tendermint:p2p","msg":"Starting MConnection service","peer":"9fbca3364885ebc82bf5be8156bb4df0a563bbf6@34.86.145.181:26656","ts":"2020-06-18T16:35:55.801790405Z"}
{"caller":"switch.go:813","level":"info","module":"tendermint:p2p","msg":"Added peer","peer":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","ts":"2020-06-18T16:35:55.801904Z"}
{"caller":"connection.go:593","conn":"MConn{34.86.145.181:26656}","level":"info","module":"tendermint:p2p","msg":"Connection is closed @ recvRoutine (likely by the other side)","peer":"9fbca3364885ebc82bf5be8156bb4df0a563bbf6@34.86.145.181:26656","ts":"2020-06-18T16:35:56.018229529Z"}
{"caller":"service.go:168","impl":"MConn{34.86.145.181:26656}","level":"info","module":"tendermint:p2p","msg":"Stopping MConnection service","peer":"9fbca3364885ebc82bf5be8156bb4df0a563bbf6@34.86.145.181:26656","ts":"2020-06-18T16:35:56.018381208Z"}
{"caller":"switch.go:324","err":"EOF","level":"error","module":"tendermint:p2p","msg":"Stopping peer for error","peer":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","ts":"2020-06-18T16:35:56.018585376Z"}
{"caller":"service.go:168","impl":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","level":"info","module":"tendermint:p2p","msg":"Stopping Peer service","peer":"9fbca3364885ebc82bf5be8156bb4df0a563bbf6@34.86.145.181:26656","ts":"2020-06-18T16:35:56.01872587Z"}
{"caller":"reactor.go:616","level":"info","module":"tendermint:consensus","msg":"Stopping gossipVotesRoutine for peer","peer":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","ts":"2020-06-18T16:35:56.103695292Z"}
{"caller":"reactor.go:474","level":"info","module":"tendermint:consensus","msg":"Stopping gossipDataRoutine for peer","peer":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","ts":"2020-06-18T16:35:56.103802702Z"}
{"caller":"reactor.go:745","level":"info","module":"tendermint:consensus","msg":"Stopping queryMaj23Routine for peer","peer":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","ts":"2020-06-18T16:35:57.811179353Z"}
{"caller":"log.go:30","level":"info","module":"grpc","msg":"transport: loopyWriter.run returning. connection error: desc = \"transport is closing\"","ts":"2020-06-18T16:35:58.245094894Z"}
{"caller":"log.go:30","level":"info","module":"grpc","msg":"transport: loopyWriter.run returning. connection error: desc = \"transport is closing\"","ts":"2020-06-18T16:35:58.25257213Z"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x1b8 pc=0xdc82e9]

goroutine 27 [running]:
github.com/oasisprotocol/oasis-core/go/consensus/tendermint/api.NewBlock(0x0, 0x1d0d940)
    github.com/oasisprotocol/oasis-core/go/consensus/tendermint/api/api.go:173 +0x29
github.com/oasisprotocol/oasis-core/go/consensus/tendermint.(*tendermintService).GetBlock(0xc006746600, 0x1d0d940, 0xc00d736750, 0x0, 0xc006746600, 0x4196cb, 0xc000032000)
    github.com/oasisprotocol/oasis-core/go/consensus/tendermint/tendermint.go:692 +0x87
github.com/oasisprotocol/oasis-core/go/consensus/api.handlerGetBlock.func1(0x1d0d940, 0xc00d736750, 0x163a500, 0x2c9b0e0, 0x10, 0xc00db356e0, 0x41aab3, 0x16f8a00)
    github.com/oasisprotocol/oasis-core/go/consensus/api/grpc.go:284 +0x89
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1(0x1d0d940, 0xc00d736750, 0x163a500, 0x2c9b0e0, 0x163a500, 0x2c9b0e0, 0x0, 0x0)
    github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/chain.go:31 +0x114
github.com/oasisprotocol/oasis-core/go/common/grpc/auth.UnaryServerInterceptor.func1(0x1d0d940, 0xc00d736750, 0x163a500, 0x2c9b0e0, 0xc00dbdeb20, 0xc00dd56d20, 0xc00d736750, 0xc00db357b8, 0x5237d8, 0x180f900)
    github.com/oasisprotocol/oasis-core/go/common/grpc/auth/auth.go:44 +0x20d
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1(0x1d0d940, 0xc00d736750, 0x163a500, 0x2c9b0e0, 0x16da320, 0x2c99760, 0x1873840, 0x2c99760)
    github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/chain.go:34 +0x9c
github.com/oasisprotocol/oasis-core/go/common/grpc.serverUnaryErrorMapper(0x1d0d940, 0xc00d736750, 0x163a500, 0x2c9b0e0, 0xc00dbdeb20, 0xc00dd56d20, 0x0, 0xc00dbdeb80, 0xf6eca99ddda03400, 0x0)
    github.com/oasisprotocol/oasis-core/go/common/grpc/errors.go:80 +0x55
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1(0x1d0d940, 0xc00d736750, 0x163a500, 0x2c9b0e0, 0xc00011dce0, 0x1e, 0x1d0d940, 0xc00d736750)
    github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/chain.go:34 +0x9c
github.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1(0x1d0d940, 0xc00d736690, 0x163a500, 0x2c9b0e0, 0xc00dbdeb20, 0xc00dd56d20, 0xc00db35970, 0x4d99d6, 0x5eeb97f4, 0x16bf7b6a)
    github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/tracing/opentracing/server_interceptors.go:30 +0x165
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1(0x1d0d940, 0xc00d736690, 0x163a500, 0x2c9b0e0, 0xc00677e088, 0xc00db359d0, 0x15f2d80, 0xc00dbdcf60)
    github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/chain.go:34 +0x9c
github.com/oasisprotocol/oasis-core/go/common/grpc.(*grpcLogAdapter).unaryLogger(0xc0011ff920, 0x1d0d940, 0xc00d736690, 0x163a500, 0x2c9b0e0, 0xc00dbdeb20, 0xc00dd56d20, 0x50, 0x183b880, 0x1, ...)
    github.com/oasisprotocol/oasis-core/go/common/grpc/grpc.go:166 +0x16b
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1(0x1d0d940, 0xc00d736690, 0x163a500, 0x2c9b0e0, 0xc00dbdeb20, 0xc00dbdeb40, 0x180f900, 0xc00dbdcf01, 0xc00d736690, 0xc00db35ba0)
    github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/chain.go:39 +0x155
github.com/oasisprotocol/oasis-core/go/consensus/api.handlerGetBlock(0x19017e0, 0xc006746600, 0x1d0d940, 0xc00d736690, 0xc00dc74ea0, 0xc0011ff950, 0x1d0d940, 0xc00d736690, 0xc00dbdcf58, 0x1)
    github.com/oasisprotocol/oasis-core/go/consensus/api/grpc.go:286 +0x197
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0002401a0, 0x1d23aa0, 0xc00db5b800, 0xc00dfe0200, 0xc00dc4d7d0, 0x2b05250, 0x0, 0x0, 0x0)
    google.golang.org/grpc@v1.29.1/server.go:1082 +0x50a
google.golang.org/grpc.(*Server).handleStream(0xc0002401a0, 0x1d23aa0, 0xc00db5b800, 0xc00dfe0200, 0x0)
    google.golang.org/grpc@v1.29.1/server.go:1405 +0xccb
google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc00dbdcf10, 0xc0002401a0, 0x1d23aa0, 0xc00db5b800, 0xc00dfe0200)
    google.golang.org/grpc@v1.29.1/server.go:746 +0xa1
created by google.golang.org/grpc.(*Server).serveStreams.func1
    google.golang.org/grpc@v1.29.1/server.go:744 +0xa1`
EXPECTED RESULTS
kostko commented 4 years ago

Thanks for your report, this is a corner case that happens during certain RPC queries while there are no blocks committed yet. Will fix this ASAP!

kostko commented 4 years ago

Proposed fix in #3033, will backport it to 20.8.x.