search

varnav / binance-node-docker

Docker image for Binance full and light nodes
MIT License
25 stars 11 forks source link

Exits 137 without output #8

Closed cbarraford closed 5 years ago

cbarraford commented 5 years ago

Trying to get a full node running and it runs for a couple minutes before it exits with 137 status code but without any interesting logs...

docker run --rm -it varnav/binance-node

Log output.

I[2019-10-28|04:03:20.935] finished write recovery chunk                module=statesync isComplete=false hash=b3cf741d7ecc3df7cc373678b1b2d5b6be745ee6c15d07da55e7d2da728648f2 startIdx=1494934 numOfNodes=781 chunkCompletion=0
I[2019-10-28|04:03:20.935] try to request state                         module=statesync peer=8c379d4d3b9995c712665dc9a9414dbde5b30483 req="46658512, 4b79d39429a225661346f1e7a860c8ac45a7f6bb43f7acc7a52b87a2ad265888"
I[2019-10-28|04:03:20.937] try to request state                         module=statesync peer=8c379d4d3b9995c712665dc9a9414dbde5b30483 req="46658512, deba802a21ad0c9e88c79a0cee2def07aaaf6dfec852464d1b9f8b42329d5ccf"
I[2019-10-28|04:03:21.016] update peer manifest hash from trustedPeers  module=statesync peer=8c379d4d3b9995c712665dc9a9414dbde5b30483
I[2019-10-28|04:03:21.017] added manifest                               module=statesync peer=8c379d4d3b9995c712665dc9a9414dbde5b30483 manifestHash=d5d22949f5da7ee074f78f94a4f9184c160bfcdf2adc9788a26f5df27b3da9d8
I[2019-10-28|04:03:21.420] processing chunk                             module=statesync peer=9612b570bffebecca4776cb4512d08e252119005 resp="[bcChunkResponseMessage 8f47d03cc0a9c75a926d0561bbfff4ba4233d4cb8e33e8d489bccf595f328c65]" numPending=145
I[2019-10-28|04:03:21.521] start write recovery chunk                   module=statesync isComplete=false hash=8f47d03cc0a9c75a926d0561bbfff4ba4233d4cb8e33e8d489bccf595f328c65 startIdx=1405040 numOfNodes=1 chunkCompletion=2
I[2019-10-28|04:03:21.528] finished write recovery chunk                module=statesync isComplete=false hash=8f47d03cc0a9c75a926d0561bbfff4ba4233d4cb8e33e8d489bccf595f328c65 startIdx=1405040 numOfNodes=1 chunkCompletion=2
I[2019-10-28|04:03:21.565] try to request state                         module=statesync peer=9612b570bffebecca4776cb4512d08e252119005 req="46658512, 0bfb8e69ed06f181ff3244c0922b0e51b9b6f0d0ada7a660c1fe7b7056998e31"
I[2019-10-28|04:03:22.928] processing chunk                             module=statesync peer=9612b570bffebecca4776cb4512d08e252119005 resp="[bcChunkResponseMessage df1e7515509ded638fc3f6731a5b6971580b38284bf6745d50c4a34fb103ab32]" numPending=144
I[2019-10-28|04:03:23.269] start write recovery chunk                   module=statesync isComplete=false hash=df1e7515509ded638fc3f6731a5b6971580b38284bf6745d50c4a34fb103ab32 startIdx=1517171 numOfNodes=1 chunkCompletion=1
I[2019-10-28|04:03:23.342] finished write recovery chunk                module=statesync isComplete=false hash=df1e7515509ded638fc3f6731a5b6971580b38284bf6745d50c4a34fb103ab32 startIdx=1517171 numOfNodes=1 chunkCompletion=1
I[2019-10-28|04:03:23.375] processing chunk                             module=statesync peer=fe5eb5a945598476abe4826a8d31b9f8da7b1a54 resp="[bcChunkResponseMessage 169815537d0747d0287863c9275ab1fed950fbc2c6be29ed55e0e00a10c97fbc]" numPending=143
I[2019-10-28|04:03:23.689] start write recovery chunk                   module=statesync isComplete=false hash=169815537d0747d0287863c9275ab1fed950fbc2c6be29ed55e0e00a10c97fbc startIdx=1470696 numOfNodes=1 chunkCompletion=2
I[2019-10-28|04:03:23.690] finished write recovery chunk                module=statesync isComplete=false hash=169815537d0747d0287863c9275ab1fed950fbc2c6be29ed55e0e00a10c97fbc startIdx=1470696 numOfNodes=1 chunkCompletion=2
I[2019-10-28|04:03:23.758] processing chunk                             module=statesync peer=7156d461742e2a1e569fd68426009c4194830c93 resp="[bcChunkResponseMessage 63951421017d03fdd6fbdfb65f32b262c17e4d26d9b7cb49be44717d8af4b3b7]" numPending=142
I[2019-10-28|04:03:23.848] try to request state                         module=statesync peer=fe5eb5a945598476abe4826a8d31b9f8da7b1a54 req="46658512, 64dd8ed2eeeb94235fd7422068af4cd99dbc28a373923fde5116469179981947"
I[2019-10-28|04:03:23.852] try to request state                         module=statesync peer=9612b570bffebecca4776cb4512d08e252119005 req="46658512, 6f5a586e3627636e40995533618a9419ef4e14fc4c5b3059ea3d5435f8ce94d9"
I[2019-10-28|04:03:23.943] start write recovery chunk                   module=statesync isComplete=false hash=63951421017d03fdd6fbdfb65f32b262c17e4d26d9b7cb49be44717d8af4b3b7 startIdx=1405128 numOfNodes=1 chunkCompletion=2
I[2019-10-28|04:03:23.943] finished write recovery chunk                module=statesync isComplete=false hash=63951421017d03fdd6fbdfb65f32b262c17e4d26d9b7cb49be44717d8af4b3b7 startIdx=1405128 numOfNodes=1 chunkCompletion=2
I[2019-10-28|04:03:23.986] processing chunk                             module=statesync peer=8c379d4d3b9995c712665dc9a9414dbde5b30483 resp="[bcChunkResponseMessage f3c90837843ae3a5202e7ebec4292b645e39bc9f7d79e676dae89d01662251f1]" numPending=141
I[2019-10-28|04:03:24.257] start write recovery chunk                   module=statesync isComplete=false hash=f3c90837843ae3a5202e7ebec4292b645e39bc9f7d79e676dae89d01662251f1 startIdx=1405043 numOfNodes=1 chunkCompletion=2
I[2019-10-28|04:03:24.306] finished write recovery chunk                module=statesync isComplete=false hash=f3c90837843ae3a5202e7ebec4292b645e39bc9f7d79e676dae89d01662251f1 startIdx=1405043 numOfNodes=1 chunkCompletion=2
I[2019-10-28|04:03:24.402] try to request state                         module=statesync peer=7156d461742e2a1e569fd68426009c4194830c93 req="46658512, bd292372a43c4a668693e78f81177647b116b6387cca1d6b8d06437fe591bcc9"
I[2019-10-28|04:03:24.403] try to request state                         module=statesync peer=8c379d4d3b9995c712665dc9a9414dbde5b30483 req="46658512, 125fbc3294960274c29d4b6adfcf17ea27a863dc80a80da8450a1b1d58a2b930"
I[2019-10-28|04:03:24.422] processing chunk                             module=statesync peer=fe5eb5a945598476abe4826a8d31b9f8da7b1a54 resp="[bcChunkResponseMessage 728cf77adc1042516ce657811cec099de88eb709b4e42d29d9effb944ed799a0]" numPending=140
cbarraford commented 5 years ago

If i run the agent a second time, here is the output.

I[2019-10-28|04:08:02.906] Starting ABCI with Tendermint                module=main
E[2019-10-28|04:08:02.993] Failed to load the latest breathe block height from module=dexkeeper timeNow=2019-10-28T04:08:02.993631Z
I[2019-10-28|04:08:02.993] Loading order book snapshot from last breathe block module=main blockHeight=0
I[2019-10-28|04:08:02.994] No breathe block is ever saved. just created match engines for all the pairs. module=main
I[2019-10-28|04:08:02.994] Initialized Block Store for replay           module=dex toHeight=0
E[2019-10-28|04:08:02.997] Failed to load the latest breathe block height from module=dexkeeper timeNow=2019-10-28T04:08:02.9969202Z
I[2019-10-28|04:08:02.997] get last breathe block height                module=main height=0
I[2019-10-28|04:08:02.997] reload latest snapshot                       module=statesync height=0
I[2019-10-28|04:08:03.074] ABCI Handshake App Info                      module=consensus height=0 hash= software-version= protocol-version=0
I[2019-10-28|04:08:03.081] ABCI Replay Blocks                           module=consensus appHeight=0 storeHeight=46658512 stateHeight=46658512
panic: Value missing for hash aea10fde6fc208fbc8099e5c4bcfc3fd71c955777d12d513d52ab456539669ac corresponding to nodeKey n���o�� �\K���q�Uw}��*�VS�i�

goroutine 1 [running]:
github.com/binance-chain/node/vendor/github.com/tendermint/iavl.(*nodeDB).GetNode(0xc000cb2ff0, 0xc000ab3160, 0x20, 0x20, 0x0)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/tendermint/iavl/nodedb.go:82 +0x617
github.com/binance-chain/node/vendor/github.com/tendermint/iavl.(*Node).getRightNode(...)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/tendermint/iavl/node.go:363
github.com/binance-chain/node/vendor/github.com/tendermint/iavl.(*Node).get(0xc000b56000, 0xc000b105c0, 0x1db8000, 0x13, 0x13, 0x1c, 0x1, 0x14abaa0, 0xc000a829d8)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/tendermint/iavl/node.go:176 +0x319
github.com/binance-chain/node/vendor/github.com/tendermint/iavl.(*Node).get(0xc0000ad1e0, 0xc000b105c0, 0x1db8000, 0x13, 0x13, 0x8, 0xc0000300c0, 0xc000c7f920, 0xc000a829d8)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/tendermint/iavl/node.go:177 +0x293
github.com/binance-chain/node/vendor/github.com/tendermint/iavl.(*ImmutableTree).Get(...)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/tendermint/iavl/immutable_tree.go:98
github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/store.(*IavlStore).Get(0xc000b100a0, 0x1db8000, 0x13, 0x13, 0x1e32fe0, 0xc00003cc00, 0x14abaa0)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/store/iavlstore.go:153 +0x86
github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/store.(*cacheKVStore).Get(0xc000b116a0, 0x1db8000, 0x13, 0x13, 0x0, 0x0, 0x0)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/store/cachekvstore.go:50 +0x125
github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/x/auth.AccountKeeper.GetNextAccountNumber(0x148bec0, 0xc00003cc50, 0x130c790, 0xc00013d2d0, 0x149aec0, 0xc000c7fe00, 0xc0008b9540, 0xb, 0x0)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/x/auth/mapper.go:162 +0xc9
github.com/binance-chain/node/app.(*BinanceChain).initChainerFn.func1(0x149aec0, 0xc000c7fe00, 0xc0008b9540, 0xb, 0x1ddb536b, 0xed4127047, 0x0, 0xc000478520, 0x12, 0xc0008b9400, ...)
    /tmp/go/src/github.com/binance-chain/node/app/app.go:364 +0x350
github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).InitChain(0xc0009a51d0, 0x1ddb536b, 0xed4127047, 0x0, 0xc000478520, 0x12, 0xc0008b9400, 0x1e31730, 0x0, 0x0, ...)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/baseapp/baseapp.go:352 +0x19f
github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/server/concurrent.(*asyncLocalClient).InitChainSync(0xc0000add90, 0x1ddb536b, 0xed4127047, 0x0, 0xc000478520, 0x12, 0xc0008b9400, 0x1e31730, 0x0, 0x0, ...)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/server/concurrent/async_local_client.go:397 +0xec
github.com/binance-chain/node/vendor/github.com/tendermint/tendermint/proxy.(*appConnConsensus).InitChainSync(0xc00032fac0, 0x1ddb536b, 0xed4127047, 0x0, 0xc000478520, 0x12, 0xc0008b9400, 0x1e31730, 0x0, 0x0, ...)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/tendermint/tendermint/proxy/app_conn.go:103 +0x6b
github.com/binance-chain/node/vendor/github.com/tendermint/tendermint/consensus.(*Handshaker).ReplayBlocks(0xc000a8c780, 0xa, 0x0, 0xc0008f6be6, 0x6, 0xc000478a80, 0x12, 0x2c7f3d0, 0x2746775, 0xc000478aa0, ...)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/tendermint/tendermint/consensus/replay.go:307 +0x68f
github.com/binance-chain/node/vendor/github.com/tendermint/tendermint/consensus.(*Handshaker).Handshake(0xc000a8c780, 0x14b0c40, 0xc0000dbea0, 0x2, 0x149ba80)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/tendermint/tendermint/consensus/replay.go:266 +0x6fa
github.com/binance-chain/node/vendor/github.com/tendermint/tendermint/node.NewNode(0xc00009c9a0, 0x149bc40, 0xc000a78820, 0xc000bcdf70, 0x147b600, 0xc000cb3040, 0xc000aa41f0, 0x130e1b8, 0xc000aa4200, 0x149ba80, ...)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/tendermint/tendermint/node/node.go:312 +0x1093
github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/server.startInProcess(0xc000467de0, 0x130ea78, 0x1d, 0x0, 0x0)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/server/start.go:133 +0x561
github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/server.StartCmd.func1(0xc000abd8c0, 0xc00035a360, 0x0, 0x2, 0x0, 0x0)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/cosmos/cosmos-sdk/server/start.go:44 +0xb5
github.com/binance-chain/node/vendor/github.com/spf13/cobra.(*Command).execute(0xc000abd8c0, 0xc00035a280, 0x2, 0x2, 0xc000abd8c0, 0xc00035a280)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/spf13/cobra/command.go:698 +0x431
github.com/binance-chain/node/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc000abc240, 0x2, 0xc00035a0c0, 0x11b25b8)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/spf13/cobra/command.go:783 +0x2ca
github.com/binance-chain/node/vendor/github.com/spf13/cobra.(*Command).Execute(...)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/spf13/cobra/command.go:736
github.com/binance-chain/node/vendor/github.com/tendermint/tendermint/libs/cli.Executor.Execute(0xc000abc240, 0x130ee80, 0x2, 0xc000ab2400)
    /tmp/go/src/github.com/binance-chain/node/vendor/github.com/tendermint/tendermint/libs/cli/setup.go:89 +0x3c
main.main()
    /tmp/go/src/github.com/binance-chain/node/cmd/bnbchaind/main.go:54 +0x3ef
cbarraford commented 5 years ago

cc @varnav

varnav commented 5 years ago

Hi! Seems to be their bug, please take a look:

https://github.com/binance-chain/node-binary/issues/108

But it works for me. Just try running again, maybe it's something like problem in the network itself?

varnav commented 5 years ago

I have a feeling that it may be also if system where you run this is not powerful enough. Make sure CPU is fast and you're running this on SSD.

varnav commented 5 years ago

Also try this:

docker run --rm -it --ulimit nofile=16000:16000 varnav/binance-node

cbarraford commented 5 years ago

i think i was hittin the ulimit, thanks!