[BUG] Receives a panic `leveldb: closed` when `Ctrl-C` seid `v5.5.2`.

maoueh commented 4 months ago

Seid version

seid version --long

``` maoueh@sei-devnet-reader-0:~$ seid version --long name: sei server_name: version: v5.5.2-fh3.0 commit: 16f73c4b9343cefae99ba56ab07b0ef233e68f42 build_tags: netgo ledger, go: go version go1.22.2 linux/amd64 build_deps: - cosmossdk.io/errors@v1.0.0 - filippo.io/edwards25519@v1.0.0-rc.1 - github.com/99designs/keyring@v1.2.1 - github.com/CosmWasm/wasmd@v0.27.0 => github.com/sei-protocol/sei-wasmd@v0.1.5 - github.com/CosmWasm/wasmvm@v1.5.2 - github.com/DataDog/zstd@v1.4.5 - github.com/VictoriaMetrics/fastcache@v1.12.1 - github.com/alitto/pond@v1.8.3 - github.com/armon/go-metrics@v0.4.1 - github.com/beorn7/perks@v1.0.1 - github.com/bgentry/speakeasy@v0.1.0 - github.com/bits-and-blooms/bitset@v1.10.0 - github.com/btcsuite/btcd@v0.22.1 - github.com/cespare/xxhash/v2@v2.2.0 - github.com/cockroachdb/errors@v1.8.1 - github.com/cockroachdb/logtags@v0.0.0-20190617123548-eb05cc24525f - github.com/cockroachdb/pebble@v0.0.0-20230928194634-aa077af62593 - github.com/cockroachdb/redact@v1.0.8 - github.com/cockroachdb/sentry-go@v0.6.1-cockroachdb.2 - github.com/cockroachdb/tokenbucket@v0.0.0-20230807174530-cc333fc44b06 - github.com/coinbase/rosetta-sdk-go@v0.7.0 - github.com/confio/ics23/go@v0.9.0 => github.com/cosmos/cosmos-sdk/ics23/go@v0.8.0 - github.com/consensys/bavard@v0.1.13 - github.com/consensys/gnark-crypto@v0.12.1 - github.com/cosmos/btcutil@v1.0.5 - github.com/cosmos/cosmos-sdk@v0.45.10 => github.com/sei-protocol/sei-cosmos@v0.3.13 - github.com/cosmos/go-bip39@v1.0.0 - github.com/cosmos/iavl@v0.21.0-alpha.1.0.20230904092046-df3db2d96583 => github.com/sei-protocol/sei-iavl@v0.1.9 - github.com/cosmos/ibc-go/v3@v3.0.0 => github.com/sei-protocol/sei-ibc-go/v3@v3.3.0 - github.com/cosmos/ledger-cosmos-go@v0.12.4 - github.com/cpuguy83/go-md2man/v2@v2.0.2 - github.com/crate-crypto/go-ipa@v0.0.0-20231025140028-3c0104f4b233 - github.com/crate-crypto/go-kzg-4844@v0.7.0 - github.com/creachadair/taskgroup@v0.3.2 - github.com/davecgh/go-spew@v1.1.1 - github.com/deckarep/golang-set@v1.8.0 - github.com/deckarep/golang-set/v2@v2.1.0 - github.com/desertbit/timer@v0.0.0-20180107155436-c41aec40b27f - github.com/dvsekhvalnov/jose2go@v1.5.0 - github.com/ethereum/go-ethereum@v1.13.2 => github.com/sei-protocol/go-ethereum@v1.13.5-sei-17 - github.com/felixge/httpsnoop@v1.0.2 - github.com/fjl/memsize@v0.0.0-20190710130421-bcb5799ab5e5 - github.com/fsnotify/fsnotify@v1.6.0 - github.com/gballet/go-libpcsclite@v0.0.0-20190607065134-2772fd86a8ff - github.com/gballet/go-verkle@v0.1.1-0.20231031103413-a67434b50f46 - github.com/go-kit/kit@v0.12.0 - github.com/go-kit/log@v0.2.1 - github.com/go-logfmt/logfmt@v0.5.1 - github.com/go-logr/logr@v1.2.3 - github.com/go-logr/stdr@v1.2.2 - github.com/godbus/dbus@v0.0.0-20190726142602-4481cbc300e2 - github.com/gofrs/flock@v0.8.1 - github.com/gogo/gateway@v1.1.0 - github.com/gogo/protobuf@v1.3.3 => github.com/regen-network/protobuf@v1.3.3-alpha.regen.1 - github.com/golang-jwt/jwt/v4@v4.5.0 - github.com/golang/protobuf@v1.5.4 - github.com/golang/snappy@v0.0.5-0.20220116011046-fa5810519dcb - github.com/google/btree@v1.1.2 - github.com/google/gofuzz@v1.2.0 - github.com/google/orderedcode@v0.0.1 - github.com/google/uuid@v1.3.0 - github.com/gorilla/handlers@v1.5.1 - github.com/gorilla/mux@v1.8.0 - github.com/gorilla/websocket@v1.5.0 - github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0 - github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0 - github.com/grpc-ecosystem/grpc-gateway@v1.16.0 - github.com/gsterjov/go-libsecret@v0.0.0-20161001094733-a6f4afe4910c - github.com/hashicorp/go-bexpr@v0.1.10 - github.com/hashicorp/go-immutable-radix@v1.3.1 - github.com/hashicorp/golang-lru@v0.5.5-0.20210104140557-80c98217689d - github.com/hashicorp/golang-lru/v2@v2.0.1 - github.com/hashicorp/hcl@v1.0.0 - github.com/hdevalence/ed25519consensus@v0.0.0-20220222234857-c00d1f31bab3 - github.com/holiman/billy@v0.0.0-20230718173358-1c7e68d277a7 - github.com/holiman/bloomfilter/v2@v2.0.3 - github.com/holiman/uint256@v1.2.4 - github.com/huin/goupnp@v1.3.0 - github.com/improbable-eng/grpc-web@v0.14.1 - github.com/jackpal/go-nat-pmp@v1.0.2 - github.com/klauspost/compress@v1.16.3 - github.com/kr/pretty@v0.3.1 - github.com/kr/text@v0.2.0 - github.com/ledgerwatch/erigon-lib@v0.0.0-20230210071639-db0e7ed11263 - github.com/lib/pq@v1.10.6 - github.com/libp2p/go-buffer-pool@v0.1.0 - github.com/magiconair/properties@v1.8.6 - github.com/mattn/go-colorable@v0.1.13 - github.com/mattn/go-isatty@v0.0.19 - github.com/mattn/go-runewidth@v0.0.13 - github.com/matttproud/golang_protobuf_extensions@v1.0.4 - github.com/mitchellh/mapstructure@v1.5.0 - github.com/mitchellh/pointerstructure@v1.2.0 - github.com/mmcloughlin/addchain@v0.4.0 - github.com/mtibben/percent@v0.2.1 - github.com/oasisprotocol/curve25519-voi@v0.0.0-20210609091139-0a56a4bca00b - github.com/olekukonko/tablewriter@v0.0.5 - github.com/pelletier/go-toml/v2@v2.0.7 - github.com/pkg/errors@v0.9.1 - github.com/pmezard/go-difflib@v1.0.0 - github.com/prometheus/client_golang@v1.14.0 - github.com/prometheus/client_model@v0.3.0 - github.com/prometheus/common@v0.42.0 - github.com/prometheus/procfs@v0.9.0 - github.com/rakyll/statik@v0.1.7 - github.com/regen-network/cosmos-proto@v0.3.1 - github.com/rivo/uniseg@v0.2.0 - github.com/rogpeppe/go-internal@v1.11.0 - github.com/rs/cors@v1.8.2 - github.com/rs/zerolog@v1.30.0 - github.com/russross/blackfriday/v2@v2.1.0 - github.com/savaki/jq@v0.0.0-20161209013833-0e6baecebbf8 - github.com/sei-protocol/goutils@v0.0.2 - github.com/sei-protocol/sei-db@v0.0.27-0.20240123064153-d6dfa112e760 => github.com/sei-protocol/sei-db@v0.0.36 - github.com/sei-protocol/sei-tm-db@v0.0.5 - github.com/shirou/gopsutil@v3.21.4-0.20210419000835-c7a38de76ee5+incompatible - github.com/spf13/afero@v1.8.2 - github.com/spf13/cast@v1.5.0 - github.com/spf13/cobra@v1.6.1 - github.com/spf13/jwalterweatherman@v1.1.0 - github.com/spf13/pflag@v1.0.5 - github.com/spf13/viper@v1.13.0 - github.com/status-im/keycard-go@v0.2.0 - github.com/stretchr/testify@v1.8.4 - github.com/subosito/gotenv@v1.4.1 - github.com/syndtr/goleveldb@v1.0.1-0.20220721030215-126854af5e6d => github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7 - github.com/tendermint/btcd@v0.1.1 - github.com/tendermint/crypto@v0.0.0-20191022145703-50d29ede1e15 - github.com/tendermint/go-amino@v0.16.0 - github.com/tendermint/tendermint@v0.37.0-dev => github.com/sei-protocol/sei-tendermint@v0.3.3 - github.com/tendermint/tm-db@v0.6.8-0.20220519162814-e24b96538a12 => github.com/sei-protocol/tm-db@v0.0.4 - github.com/tidwall/gjson@v1.10.2 - github.com/tidwall/match@v1.1.1 - github.com/tidwall/pretty@v1.2.0 - github.com/tidwall/tinylru@v1.1.0 - github.com/tidwall/wal@v1.1.7 - github.com/tklauser/go-sysconf@v0.3.12 - github.com/tklauser/numcpus@v0.6.1 - github.com/tyler-smith/go-bip39@v1.1.0 - github.com/urfave/cli/v2@v2.25.7 - github.com/xrash/smetrics@v0.0.0-20201216005158-039620a65673 - github.com/yourbasic/graph@v0.0.0-20210606180040-8ecfec1c2869 - github.com/zbiljic/go-filelock@v0.0.0-20170914061330-1dbf7103ab7d - github.com/zondax/hid@v0.9.2 - github.com/zondax/ledger-go@v0.14.3 - go.opentelemetry.io/otel@v1.9.0 - go.opentelemetry.io/otel/exporters/jaeger@v1.9.0 - go.opentelemetry.io/otel/sdk@v1.9.0 - go.opentelemetry.io/otel/trace@v1.9.0 - golang.org/x/crypto@v0.23.0 - golang.org/x/exp@v0.0.0-20231110203233-9a3e6036ecaa - golang.org/x/net@v0.25.0 - golang.org/x/sync@v0.7.0 - golang.org/x/sys@v0.20.0 - golang.org/x/term@v0.20.0 - golang.org/x/text@v0.15.0 - golang.org/x/time@v0.3.0 - google.golang.org/genproto@v0.0.0-20240116215550-a9fa1716bcac - google.golang.org/genproto/googleapis/api@v0.0.0-20240125205218-1f4bbc51befe - google.golang.org/genproto/googleapis/rpc@v0.0.0-20240125205218-1f4bbc51befe - google.golang.org/grpc@v1.61.0 => google.golang.org/grpc@v1.33.2 - google.golang.org/protobuf@v1.33.0 - gopkg.in/ini.v1@v1.67.0 - gopkg.in/natefinch/lumberjack.v2@v2.0.0 - gopkg.in/yaml.v2@v2.4.0 - gopkg.in/yaml.v3@v3.0.1 - nhooyr.io/websocket@v1.8.6 - rsc.io/tmplfunc@v0.0.3 cosmos_sdk_version: v0.45.10 ```

Chain ID pacific-1, just after the v5.5.2 upgrade

Describe the bug Receives a panic leveldb: closed when Ctrl-C seid v5.5.2.

To Reproduce Steps to reproduce the behavior:

I ranseid via systemctl
At some point I do systemctl stop <name>.
Receive the following panic:

...
5:07PM ERR problem closing blockstore err="leveldb: closed"
5:07PM ERR problem closing statestore err="leveldb: closed"
5:07PM INF stopped service service=Node
5:07PM INF RPC HTTP server stopped err="accept tcp [::]:1317: use of closed network connection" module=api-server
5:07PM INF close any other open resource...
5:07PM INF committed state block_app_hash=F7D6B6CC5818F999C6816F89C3785927618B45A5E3CA6A9E9D39360753E9000E height=79124790 module=state num_txs=31
5:07PM INF transaction expired address= age=30126.250199 evm=false height=79124790 module=mempool nonce=0 priority=0 tx=5BF3DD1E0B0CF2D63E40089A54C255DFA3BEB5D53E907C7D06106FEB17E6EBAD tx_height=79124571 tx_timestamp=2024-05-27T17:07:25Z
panic: failed to process committed block (79124790:3F5FAD6845CC289554CE414176312D32E7B5D7344D9FE0A17AB98D97F1FE0A4C): leveldb: closed
goroutine 442 [running]:
github.com/tendermint/tendermint/internal/blocksync.(*Reactor).poolRoutine(0xc0064f18c8, {0x3655f40, 0xc000398050}, 0x0, 0xc019fff3e0)
    /home/sei/go/pkg/mod/github.com/sei-protocol/sei-tendermint@v0.3.3/internal/blocksync/reactor.go:700 +0x1b9a
created by github.com/tendermint/tendermint/internal/blocksync.(*Reactor).OnStart in goroutine 1
    /home/sei/go/pkg/mod/github.com/sei-protocol/sei-tendermint@v0.3.3/internal/blocksync/reactor.go:181 +0x5d7

Note that it's not impossible that multiple SIGTERM has been sent

Expected behavior No panic

maoueh commented 4 months ago

See full logs

``` 5:07PM INF stopping service module=p2p peer={"Hostname":"144.76.108.94","NodeID":"63fc8dd6ea5c7afd63666cb75a35cc646988ae47","Path":"","Port":26656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=BlockSync 5:07PM INF stopping service service=Consensus 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service service=Consensus 5:07PM INF stopping service module=evidence service=Evidence 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=p2p peer={"Hostname":"144.76.108.94","NodeID":"63fc8dd6ea5c7afd63666cb75a35cc646988ae47","Path":"","Port":26656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=BlockPool 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=p2p peer={"Hostname":"213.190.31.21","NodeID":"99d8cd3e400284d907a09a70217f6eda236391eb","Path":"","Port":11956,"Protocol":"mconn"} service=MConnection 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=p2p peer={"Hostname":"135.181.238.250","NodeID":"d863f6a2e6cae0ec2146d89905af97b71c15b657","Path":"","Port":26656,"Protocol":"mconn"} service=MConnection 5:07PM ERR Subscription terminated err="context canceled" module=rpc 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=evidence service=Evidence 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=p2p peer={"Hostname":"135.181.238.250","NodeID":"d863f6a2e6cae0ec2146d89905af97b71c15b657","Path":"","Port":26656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service service=PEX 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service service=PEX 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=p2p peer={"Hostname":"52.195.241.204","NodeID":"9ebc2f815ef66ec895f8d766e927be69aae0c336","Path":"","Port":26656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=p2p peer={"Hostname":"167.235.117.26","NodeID":"03cec5f8ed2c7c16b3b285aeb952460c2122ed0e","Path":"","Port":11956,"Protocol":"mconn"} service=MConnection 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=p2p peer={"Hostname":"213.190.31.21","NodeID":"99d8cd3e400284d907a09a70217f6eda236391eb","Path":"","Port":11956,"Protocol":"mconn"} service=MConnection 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=p2p peer={"Hostname":"167.235.117.26","NodeID":"03cec5f8ed2c7c16b3b285aeb952460c2122ed0e","Path":"","Port":11956,"Protocol":"mconn"} service=MConnection 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=p2p peer={"Hostname":"5.9.102.218","NodeID":"7c1d748a557a52ceee8a94d0b1ee5e2dc708fe42","Path":"","Port":26656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=p2p peer={"Hostname":"5.9.102.218","NodeID":"7c1d748a557a52ceee8a94d0b1ee5e2dc708fe42","Path":"","Port":26656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopping service module=p2p peer={"Hostname":"135.125.189.10","NodeID":"7c91f986605d4f68d94a4ecde7ffe0d8437257f2","Path":"","Port":51656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=p2p peer={"Hostname":"54.193.193.143","NodeID":"bbf11b8b91d79763624540ffa511dfdc5bcb4a2e","Path":"","Port":26656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=txindex service=IndexerService 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=p2p peer={"Hostname":"18.136.181.19","NodeID":"61c6035b61100f770b66cecf4dcf212b05dd3dfb","Path":"","Port":26656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=eventbus service=EventBus 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=p2p peer={"Hostname":"135.125.189.10","NodeID":"7c91f986605d4f68d94a4ecde7ffe0d8437257f2","Path":"","Port":51656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=eventbus service=EventBus 5:07PM INF stopping service service=Node 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=statesync service=StateSync 5:07PM INF stopped service module=statesync service=StateSync 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=p2p peer={"Hostname":"52.195.241.204","NodeID":"9ebc2f815ef66ec895f8d766e927be69aae0c336","Path":"","Port":26656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM ERR subscription was canceled, resubscribing err="subscription terminated by publisher" query="tm.event = 'NewBlockHeader'" 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM ERR Prometheus HTTP server ListenAndServe err="http: Server closed" 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF RPC HTTP server stopped err="http: Server closed" module=rpc-server 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=p2p peer={"Hostname":"18.136.181.19","NodeID":"61c6035b61100f770b66cecf4dcf212b05dd3dfb","Path":"","Port":26656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=mempool service=Mempool 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=mempool service=Mempool 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM ERR error serving server err="http: Server closed" module=rpc 5:07PM INF stopping service module=dbsync service=DBSync 5:07PM INF stopping service service=localClient 5:07PM INF stopped service module=dbsync service=DBSync 5:07PM INF stopped service service=localClient 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF Stopping Node 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF Closing rpc listener listener={"Listener":{}} 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM ERR error closing listener err="close tcp 127.0.0.1:26657: use of closed network connection" listener={"Listener":{}} 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM ERR failed to stop event sink err="leveldb: closed" 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=txindex service=IndexerService 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=events service=PubSub 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=events service=PubSub 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=proxy service=proxyClient 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=proxy service=proxyClient 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=p2p peer={"Hostname":"54.193.193.143","NodeID":"bbf11b8b91d79763624540ffa511dfdc5bcb4a2e","Path":"","Port":26656,"Protocol":"mconn"} service=MConnection 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=p2p service=router 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM ERR failed to close transport err="close tcp [::]:26656: use of closed network connection" module=p2p 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=p2p service=router 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM ERR peer failure endpoint={} err="context canceled" module=p2p peer=63fc8dd6ea5c7afd63666cb75a35cc646988ae47 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=BlockPool 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF peer disconnected endpoint={} module=p2p peer=99d8cd3e400284d907a09a70217f6eda236391eb 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF peer disconnected endpoint={} module=p2p peer=d863f6a2e6cae0ec2146d89905af97b71c15b657 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF peer disconnected endpoint={} module=p2p peer=03cec5f8ed2c7c16b3b285aeb952460c2122ed0e 5:07PM INF stopped service module=blockchain service=BlockSync 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF peer disconnected endpoint={} module=p2p peer=7c1d748a557a52ceee8a94d0b1ee5e2dc708fe42 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM ERR peer failure endpoint={} err="context canceled" module=p2p peer=7c91f986605d4f68d94a4ecde7ffe0d8437257f2 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM ERR peer failure endpoint={} err="context canceled" module=p2p peer=bbf11b8b91d79763624540ffa511dfdc5bcb4a2e 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM ERR peer failure endpoint={} err="context canceled" module=p2p peer=9ebc2f815ef66ec895f8d766e927be69aae0c336 5:07PM INF stopping service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM INF stopped service module=blockchain service=bpRequester 5:07PM ERR peer failure endpoint={} err="context canceled" module=p2p peer=61c6035b61100f770b66cecf4dcf212b05dd3dfb 5:07PM ERR problem closing blockstore err="leveldb: closed" 5:07PM ERR problem closing statestore err="leveldb: closed" 5:07PM INF stopped service service=Node 5:07PM INF RPC HTTP server stopped err="accept tcp [::]:1317: use of closed network connection" module=api-server 5:07PM INF close any other open resource... 5:07PM INF committed state block_app_hash=F7D6B6CC5818F999C6816F89C3785927618B45A5E3CA6A9E9D39360753E9000E height=79124790 module=state num_txs=31 5:07PM INF transaction expired address= age=30126.250199 evm=false height=79124790 module=mempool nonce=0 priority=0 tx=5BF3DD1E0B0CF2D63E40089A54C255DFA3BEB5D53E907C7D06106FEB17E6EBAD tx_height=79124571 tx_timestamp=2024-05-27T17:07:25Z panic: failed to process committed block (79124790:3F5FAD6845CC289554CE414176312D32E7B5D7344D9FE0A17AB98D97F1FE0A4C): leveldb: closed goroutine 442 [running]: github.com/tendermint/tendermint/internal/blocksync.(*Reactor).poolRoutine(0xc0064f18c8, {0x3655f40, 0xc000398050}, 0x0, 0xc019fff3e0) /home/sei/go/pkg/mod/github.com/sei-protocol/sei-tendermint@v0.3.3/internal/blocksync/reactor.go:700 +0x1b9a created by github.com/tendermint/tendermint/internal/blocksync.(*Reactor).OnStart in goroutine 1 /home/sei/go/pkg/mod/github.com/sei-protocol/sei-tendermint@v0.3.3/internal/blocksync/reactor.go:181 +0x5d7 ```

philipsu522 commented 3 months ago

@maoueh are you still running into issues here?

maoueh commented 2 months ago

I haven't restarted my nodes recently. I'll try this week to perform some restart and check how it looks like.

sei-protocol / sei-chain

[BUG] Receives a panic `leveldb: closed` when `Ctrl-C` seid `v5.5.2`. #1704