search

lightningnetwork / lnd

Lightning Network Daemon ⚡️
MIT License
7.73k stars 2.09k forks source link

Lnd crashes on FilteredChainView #3267

Closed tomasvanagas closed 5 years ago

tomasvanagas commented 5 years ago

Background

Lnd crashes on startup after unlocking wallet. I've tried to update lnd but still the same.

Your environment

lnd verson: lnd version 0.7.0-beta commit=v0.7.0-beta-4-ga6e77811b1578f5d48191d0b9a5d4cbf895626f5

os version: Linux laiptine3 4.18.0-25-generic #26~18.04.1-Ubuntu SMP Thu Jun 27 07:28:31 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

bitcoind version: Bitcoin Core Daemon version v0.18.0.0-g2472733a24a9364e4c6233ccd04166a26a68cc65

Steps to reproduce

Lnd crashes after starting lnd and unlocking lnd wallet on "FilteredChainView starting".

Expected behaviour

Should not crash

Actual behaviour

2019-07-03 09:40:29.794 [INF] LTND: Version: 0.7.0-beta commit=v0.7.0-beta-4-ga6e77811b1578f5d48191d0b9a5d4cbf895626f5, build=production, logging=default 2019-07-03 09:40:29.794 [INF] LTND: Active chain: Bitcoin (network=mainnet) 2019-07-03 09:40:29.795 [INF] CHDB: Checking for schema update: latest_version=9, db_version=9 2019-07-03 09:40:29.807 [INF] RPCS: password gRPC proxy started at 127.0.0.1:8080 2019-07-03 09:40:29.807 [INF] RPCS: password RPC server listening on 127.0.0.1:10009 2019-07-03 09:40:29.807 [INF] LTND: Waiting for wallet encryption password. Use lncli create to create a wallet, lncli unlock to unlock an existing wallet, or lncli changepassword to change the password of an existing wallet and unlock it. 2019-07-03 09:40:42.631 [INF] LNWL: Opened wallet 2019-07-03 09:40:42.729 [INF] LTND: Primary chain is set to: bitcoin 2019-07-03 09:40:42.764 [INF] LTND: Initializing bitcoind backed fee estimator 2019-07-03 09:40:42.764 [INF] LNWL: Started listening for bitcoind block notifications via ZMQ on 127.0.0.1:28332 2019-07-03 09:40:42.764 [INF] LNWL: Started listening for bitcoind transaction notifications via ZMQ on 127.0.0.1:28333 2019-07-03 09:40:43.829 [INF] LNWL: The wallet has been unlocked without a time limit 2019-07-03 09:40:43.831 [INF] LTND: LightningWallet opened 2019-07-03 09:40:43.913 [INF] HSWC: Restoring in-memory circuit state from disk 2019-07-03 09:40:43.971 [INF] HSWC: Payment circuits loaded: num_pending=0, num_open=0 2019-07-03 09:40:43.972 [INF] HSWC: Trimming open circuits for chan_id=582646:826:1, start_htlc_id=3 2019-07-03 09:40:44.233 [INF] LTND: Channel backup proxy channel notifier starting 2019-07-03 09:40:44.234 [INF] ATPL: Instantiating autopilot with cfg: (*lnd.autoPilotConfig)(0xc0000b1480)({ Active: (bool) true, Heuristic: (map[string]float64) (len=1) { (string) (len=12) "preferential": (float64) 1 }, MaxChannels: (int) 20, Allocation: (float64) 0.5, MinChannelSize: (int64) 20000, MaxChannelSize: (int64) 16777215, Private: (bool) false, MinConfs: (int32) 0, ConfTarget: (uint32) 3 })

2019-07-03 09:40:44.235 [INF] RPCS: gRPC proxy started at 127.0.0.1:8080 2019-07-03 09:40:44.235 [INF] RPCS: RPC server listening on 127.0.0.1:10009 2019-07-03 09:40:44.236 [INF] LTND: Waiting for chain backend to finish sync, start_height=583634 2019-07-03 09:40:44.893 [INF] LNWL: Started rescan from block 00000000000000000007cd168c6f79ae9225de936419a4a1a8c789e9e9e9ed63 (height 583632) for 16 addresses 2019-07-03 09:40:45.414 [INF] LNWL: Catching up block hashes to height 583634, this might take a while 2019-07-03 09:40:45.447 [INF] LNWL: Done catching up block hashes 2019-07-03 09:40:45.447 [INF] LNWL: Finished rescan for 16 addresses (synced to block 0000000000000000000a5e93d17749609e02aba0a3ab8d2ffe71853773c820d2, height 583634) 2019-07-03 09:40:46.242 [INF] LTND: Chain backend is fully synced (end_height=583634)! 2019-07-03 09:40:46.271 [INF] HSWC: Starting HTLC Switch 2019-07-03 09:40:46.272 [INF] NTFN: New block epoch subscription 2019-07-03 09:40:46.272 [INF] NTFN: New block epoch subscription 2019-07-03 09:40:46.314 [INF] NTFN: New block epoch subscription 2019-07-03 09:40:46.315 [INF] NTFN: New block epoch subscription 2019-07-03 09:40:46.318 [INF] CNCT: Creating ChannelArbitrators for 1 active channels 2019-07-03 09:40:46.318 [INF] NTFN: New block epoch subscription 2019-07-03 09:40:46.319 [INF] NTFN: New spend subscription: spend_id=1, outpoint=1fb2878cdc05040fca8e8aab599dd4354f90f77e95156fdb4f23d56be9f5fdb6:1, height_hint=583632 2019-07-03 09:40:46.388 [INF] CNCT: Close observer for ChannelPoint(1fb2878cdc05040fca8e8aab599dd4354f90f77e95156fdb4f23d56be9f5fdb6:1) active 2019-07-03 09:40:46.388 [INF] CNCT: ChannelArbitrator(1fb2878cdc05040fca8e8aab599dd4354f90f77e95156fdb4f23d56be9f5fdb6:1): starting state=StateDefault 2019-07-03 09:40:46.406 [INF] DISC: Authenticated Gossiper is starting 2019-07-03 09:40:46.406 [INF] BRAR: Starting contract observer, watching for breaches. 2019-07-03 09:40:46.407 [INF] NTFN: New block epoch subscription 2019-07-03 09:40:46.409 [INF] CRTR: FilteredChainView starting unexpected fault address 0x7f00dc8de008 fatal error: fault [signal SIGBUS: bus error code=0x2 addr=0x7f00dc8de008 pc=0x680faa]

goroutine 1 [running]: runtime.throw(0x11af673, 0x5) /snap/go/3947/src/runtime/panic.go:617 +0x72 fp=0xc000305638 sp=0xc000305608 pc=0x42f172 runtime.sigpanic() /snap/go/3947/src/runtime/signal_unix.go:387 +0x47e fp=0xc000305668 sp=0xc000305638 pc=0x44475e github.com/coreos/bbolt.(elemRef).isLeaf(...) /home/aula/go/pkg/mod/github.com/coreos/bbolt@v1.3.2/cursor.go:387 github.com/coreos/bbolt.(Cursor).first(0xc0003057c8) /home/aula/go/pkg/mod/github.com/coreos/bbolt@v1.3.2/cursor.go:170 +0x1aa fp=0xc0003056c8 sp=0xc000305668 pc=0x680faa github.com/coreos/bbolt.(Cursor).next(0xc0003057c8, 0xc000872e00, 0xe11fc7956e243613, 0x7021a089e62a5950, 0x6c8d5712b2fd4e16, 0x16deed8b911dbb5f, 0x0, 0x4e20) /home/aula/go/pkg/mod/github.com/coreos/bbolt@v1.3.2/cursor.go:234 +0x8b fp=0xc000305718 sp=0xc0003056c8 pc=0x68127b github.com/coreos/bbolt.(Cursor).Next(0xc0003057c8, 0x24, 0x24, 0x7f00dd4deb48, 0x8, 0x8, 0x0) /home/aula/go/pkg/mod/github.com/coreos/bbolt@v1.3.2/cursor.go:75 +0x42 fp=0xc000305788 sp=0xc000305718 pc=0x680632 github.com/coreos/bbolt.(Bucket).ForEach(0xc000dcc280, 0xc000305838, 0xa, 0xa) /home/aula/go/pkg/mod/github.com/coreos/bbolt@v1.3.2/bucket.go:387 +0xb4 fp=0xc0003057f8 sp=0xc000305788 pc=0x67e5a4 github.com/lightningnetwork/lnd/channeldb.(ChannelGraph).ChannelView.func1(0xc0002db340, 0x132e6a8, 0xc0002db340) /home/aula/go/src/github.com/lightningnetwork/lnd/channeldb/graph.go:3029 +0x136 fp=0xc000305860 sp=0xc0003057f8 pc=0xa105a6 github.com/coreos/bbolt.(DB).View(0xc000474a00, 0xc0003058d0, 0x0, 0x0) /home/aula/go/pkg/mod/github.com/coreos/bbolt@v1.3.2/db.go:719 +0x90 fp=0xc0003058b0 sp=0xc000305860 pc=0x6843d0 github.com/lightningnetwork/lnd/channeldb.(ChannelGraph).ChannelView(0xc00031c270, 0xc00046cd20, 0x0, 0x0, 0x0, 0x0) /home/aula/go/src/github.com/lightningnetwork/lnd/channeldb/graph.go:3009 +0x6b fp=0xc000305908 sp=0xc0003058b0 pc=0x9edc1b github.com/lightningnetwork/lnd/routing.(ChannelRouter).Start(0xc0000b46e0, 0x0, 0x0) /home/aula/go/src/github.com/lightningnetwork/lnd/routing/router.go:495 +0x548 fp=0xc0003059f8 sp=0xc000305908 pc=0xb40ee8 github.com/lightningnetwork/lnd.(server).Start.func1() /home/aula/go/src/github.com/lightningnetwork/lnd/server.go:1197 +0x4e1 fp=0xc000305af0 sp=0xc0003059f8 pc=0xeb4571 sync.(Once).Do(0xc00012cb48, 0xc0002f9b40) /snap/go/3947/src/sync/once.go:44 +0xb3 fp=0xc000305b20 sp=0xc000305af0 pc=0x474863 github.com/lightningnetwork/lnd.(server).Start(0xc00012cb40, 0x11e4dae, 0x2e) /home/aula/go/src/github.com/lightningnetwork/lnd/server.go:1125 +0x6c fp=0xc000305b68 sp=0xc000305b20 pc=0xe91a7c github.com/lightningnetwork/lnd.Main(0x0, 0x0) /home/aula/go/src/github.com/lightningnetwork/lnd/lnd.go:473 +0x11d3 fp=0xc000305f38 sp=0xc000305b68 pc=0xe56443 main.main() /home/aula/go/src/github.com/lightningnetwork/lnd/cmd/lnd/main.go:14 +0x26 fp=0xc000305f98 sp=0xc000305f38 pc=0xec2d86 runtime.main() /snap/go/3947/src/runtime/proc.go:200 +0x20c fp=0xc000305fe0 sp=0xc000305f98 pc=0x430adc runtime.goexit() /snap/go/3947/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc000305fe8 sp=0xc000305fe0 pc=0x45e3c1

goroutine 86 [select]: google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc0006f2700) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/balancer_conn_wrappers.go:122 +0x110 created by google.golang.org/grpc.newCCBalancerWrapper /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/balancer_conn_wrappers.go:113 +0x14f

goroutine 22 [syscall]: os/signal.signal_recv(0x0) /snap/go/3947/src/runtime/sigqueue.go:139 +0x9c os/signal.loop() /snap/go/3947/src/os/signal/signal_unix.go:23 +0x22 created by os/signal.init.0 /snap/go/3947/src/os/signal/signal_unix.go:29 +0x41

goroutine 24 [select]: github.com/lightningnetwork/lnd/signal.mainInterruptHandler() /home/aula/go/src/github.com/lightningnetwork/lnd/signal/signal.go:70 +0x106 created by github.com/lightningnetwork/lnd/signal.init.1 /home/aula/go/src/github.com/lightningnetwork/lnd/signal/signal.go:38 +0xee

goroutine 29 [select]: io.(pipe).Read(0xc0000d5540, 0xc000312000, 0x1000, 0x1000, 0x1330640, 0xc00031bda8, 0xc00031bd98) /snap/go/3947/src/io/pipe.go:50 +0xe7 io.(PipeReader).Read(0xc0000ac378, 0xc000312000, 0x1000, 0x1000, 0xc0000af800, 0x1cf52db, 0x1) /snap/go/3947/src/io/pipe.go:127 +0x4c bufio.(Reader).fill(0xc00031bf50) /snap/go/3947/src/bufio/bufio.go:100 +0x10f bufio.(Reader).ReadSlice(0xc00031bf50, 0xa, 0x1, 0x0, 0x0, 0x0, 0x0) /snap/go/3947/src/bufio/bufio.go:356 +0x3d bufio.(Reader).ReadLine(0xc00031bf50, 0x1cf52db, 0x1, 0x1, 0x1, 0x0, 0x0) /snap/go/3947/src/bufio/bufio.go:385 +0x34 github.com/jrick/logrotate/rotator.(Rotator).Run(0xc000078280, 0x1487e80, 0xc0000ac378, 0xc000078280, 0x0) /home/aula/go/pkg/mod/github.com/jrick/logrotate@v1.0.0/rotator/rotator.go:100 +0x91 created by github.com/lightningnetwork/lnd.initLogRotator /home/aula/go/src/github.com/lightningnetwork/lnd/log.go:185 +0x31a

goroutine 58 [IO wait]: internal/poll.runtime_pollWait(0x7f01036ac7e0, 0x72, 0x0) /snap/go/3947/src/runtime/netpoll.go:182 +0x56 internal/poll.(pollDesc).wait(0xc00019a018, 0x72, 0x0, 0x0, 0x11b1850) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:87 +0x9b internal/poll.(pollDesc).waitRead(...) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(FD).Accept(0xc00019a000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /snap/go/3947/src/internal/poll/fd_unix.go:384 +0x1ba net.(netFD).accept(0xc00019a000, 0xc000087860, 0x146d690, 0x1) /snap/go/3947/src/net/fd_unix.go:238 +0x42 net.(TCPListener).accept(0xc000010050, 0x146d690, 0x2, 0x1) /snap/go/3947/src/net/tcpsock_posix.go:139 +0x32 net.(TCPListener).Accept(0xc000010050, 0xc0000c6770, 0x2, 0x0, 0x1) /snap/go/3947/src/net/tcpsock.go:260 +0x48 github.com/lightningnetwork/lnd/brontide.(*Listener).listen(0xc000252f00) /home/aula/go/src/github.com/lightningnetwork/lnd/brontide/listener.go:79 +0xc3 created by github.com/lightningnetwork/lnd/brontide.NewListener /home/aula/go/src/github.com/lightningnetwork/lnd/brontide/listener.go:61 +0x1b5

goroutine 56 [select]: github.com/btcsuite/btcwallet/wallet.(Wallet).rescanRPCHandler(0xc000442000) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/wallet/rescan.go:249 +0x13e created by github.com/btcsuite/btcwallet/wallet.(Wallet).SynchronizeRPC /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/wallet/wallet.go:194 +0x1b9

goroutine 54 [select]: github.com/btcsuite/btcwallet/wallet.(Wallet).rescanBatchHandler(0xc000442000) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/wallet/rescan.go:116 +0x188 created by github.com/btcsuite/btcwallet/wallet.(Wallet).SynchronizeRPC /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/wallet/wallet.go:192 +0x175

goroutine 60 [select]: github.com/lightningnetwork/lnd/queue.(*GCQueue).queueManager(0xc0002f20f0) /home/aula/go/src/github.com/lightningnetwork/lnd/queue/gc_queue.go:139 +0x1ce created by github.com/lightningnetwork/lnd/queue.NewGCQueue /home/aula/go/src/github.com/lightningnetwork/lnd/queue/gc_queue.go:76 +0x189

goroutine 55 [select]: github.com/btcsuite/btcwallet/wallet.(Wallet).rescanProgressHandler(0xc000442000) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/wallet/rescan.go:211 +0x13c created by github.com/btcsuite/btcwallet/wallet.(Wallet).SynchronizeRPC /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/wallet/wallet.go:193 +0x197

goroutine 46 [select]: github.com/btcsuite/btcwallet/wallet.(Wallet).txCreator(0xc000442000) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/wallet/wallet.go:1160 +0x106 created by github.com/btcsuite/btcwallet/wallet.(Wallet).Start /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/wallet/wallet.go:147 +0xcd

goroutine 57 [select]: github.com/lightningnetwork/lnd/lnwallet.(LightningWallet).requestHandler(0xc000130500) /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/wallet.go:373 +0xb5 created by github.com/lightningnetwork/lnd/lnwallet.(LightningWallet).Startup /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/wallet.go:313 +0xb8

goroutine 66 [IO wait]: internal/poll.runtime_pollWait(0x7f01036ac710, 0x72, 0xffffffffffffffff) /snap/go/3947/src/runtime/netpoll.go:182 +0x56 internal/poll.(pollDesc).wait(0xc00019a398, 0x72, 0x0, 0x1, 0xffffffffffffffff) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:87 +0x9b internal/poll.(pollDesc).waitRead(...) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(FD).Read(0xc00019a380, 0xc000d1404d, 0x1, 0x1, 0x0, 0x0, 0x0) /snap/go/3947/src/internal/poll/fd_unix.go:169 +0x19b net.(netFD).Read(0xc00019a380, 0xc000d1404d, 0x1, 0x1, 0x4bf0fa, 0xc00019a380, 0x0) /snap/go/3947/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc000010110, 0xc000d1404d, 0x1, 0x1, 0x0, 0x0, 0x0) /snap/go/3947/src/net/net.go:177 +0x69 io.ReadAtLeast(0x1487fc0, 0xc000010110, 0xc000d1404d, 0x1, 0x1, 0x1, 0x1df0120, 0x1487f80, 0xc0002f3090) /snap/go/3947/src/io/io.go:310 +0x88 io.ReadFull(...) /snap/go/3947/src/io/io.go:329 github.com/lightninglabs/gozmq.(Conn).readFrame(0xc000252ed0, 0xc000319e00, 0x0, 0x0, 0x0, 0x1487f80, 0xc0002f3090) /home/aula/go/pkg/mod/github.com/lightninglabs/gozmq@v0.0.0-20180324010646-462a8a753885/zmq.go:263 +0x102 github.com/lightninglabs/gozmq.(Conn).readMessage(0xc000252ed0, 0xc000020000, 0x1034560, 0x1045a40, 0x7f0103658e38, 0xc0002f3090) /home/aula/go/pkg/mod/github.com/lightninglabs/gozmq@v0.0.0-20180324010646-462a8a753885/zmq.go:312 +0x55 github.com/lightninglabs/gozmq.(Conn).Receive(0xc000252ed0, 0xc000086420, 0xc0002f3000, 0x7f0103658b60, 0xc0002f3090, 0xc0002f3001) /home/aula/go/pkg/mod/github.com/lightninglabs/gozmq@v0.0.0-20180324010646-462a8a753885/zmq.go:378 +0x40 github.com/btcsuite/btcwallet/chain.(BitcoindConn).txEventHandler(0xc00046a000, 0xc000252ed0) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_conn.go:260 +0x17a created by github.com/btcsuite/btcwallet/chain.(BitcoindConn).Start /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_conn.go:140 +0x57b

goroutine 67 [select]: github.com/btcsuite/btcd/rpcclient.(Client).sendPostHandler(0xc0000b4500) /home/aula/go/pkg/mod/github.com/btcsuite/btcd@v0.0.0-20190629003639-c26ffa870fd8/rpcclient/infrastructure.go:736 +0xd9 created by github.com/btcsuite/btcd/rpcclient.(Client).start /home/aula/go/pkg/mod/github.com/btcsuite/btcd@v0.0.0-20190629003639-c26ffa870fd8/rpcclient/infrastructure.go:1029 +0x10f

goroutine 73 [select]: github.com/btcsuite/btcwallet/chain.(ConcurrentQueue).Start.func1(0xc0003383e0) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/queue.go:55 +0x2b6 created by github.com/btcsuite/btcwallet/chain.(ConcurrentQueue).Start /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/queue.go:47 +0x3f

goroutine 12 [select]: github.com/btcsuite/btcd/rpcclient.(Client).sendPostHandler(0xc00001c0a0) /home/aula/go/pkg/mod/github.com/btcsuite/btcd@v0.0.0-20190629003639-c26ffa870fd8/rpcclient/infrastructure.go:736 +0xd9 created by github.com/btcsuite/btcd/rpcclient.(Client).start /home/aula/go/pkg/mod/github.com/btcsuite/btcd@v0.0.0-20190629003639-c26ffa870fd8/rpcclient/infrastructure.go:1029 +0x10f

goroutine 59 [select]: github.com/lightningnetwork/lnd/queue.(*GCQueue).queueManager(0xc0002f2050) /home/aula/go/src/github.com/lightningnetwork/lnd/queue/gc_queue.go:139 +0x1ce created by github.com/lightningnetwork/lnd/queue.NewGCQueue /home/aula/go/src/github.com/lightningnetwork/lnd/queue/gc_queue.go:76 +0x189

goroutine 49 [IO wait]: internal/poll.runtime_pollWait(0x7f01036ac640, 0x72, 0xffffffffffffffff) /snap/go/3947/src/runtime/netpoll.go:182 +0x56 internal/poll.(pollDesc).wait(0xc0001b4218, 0x72, 0x0, 0x1, 0xffffffffffffffff) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:87 +0x9b internal/poll.(pollDesc).waitRead(...) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(FD).Read(0xc0001b4200, 0xc000d1404c, 0x1, 0x1, 0x0, 0x0, 0x0) /snap/go/3947/src/internal/poll/fd_unix.go:169 +0x19b net.(netFD).Read(0xc0001b4200, 0xc000d1404c, 0x1, 0x1, 0x4bf0fa, 0xc0001b4200, 0x0) /snap/go/3947/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc000010100, 0xc000d1404c, 0x1, 0x1, 0x0, 0x0, 0x0) /snap/go/3947/src/net/net.go:177 +0x69 io.ReadAtLeast(0x1487fc0, 0xc000010100, 0xc000d1404c, 0x1, 0x1, 0x1, 0x1df0120, 0x1487f80, 0xc0002f2fa0) /snap/go/3947/src/io/io.go:310 +0x88 io.ReadFull(...) /snap/go/3947/src/io/io.go:329 github.com/lightninglabs/gozmq.(Conn).readFrame(0xc000252d50, 0xc0000d5500, 0x0, 0x0, 0x0, 0x1487f80, 0xc0002f2fa0) /home/aula/go/pkg/mod/github.com/lightninglabs/gozmq@v0.0.0-20180324010646-462a8a753885/zmq.go:263 +0x102 github.com/lightninglabs/gozmq.(Conn).readMessage(0xc000252d50, 0xc000020000, 0x1034560, 0x1045a40, 0x7f0103658e38, 0xc0002f2fa0) /home/aula/go/pkg/mod/github.com/lightninglabs/gozmq@v0.0.0-20180324010646-462a8a753885/zmq.go:312 +0x55 github.com/lightninglabs/gozmq.(Conn).Receive(0xc000252d50, 0xc000086420, 0xc0002f2f00, 0x7f0103658b60, 0xc0002f2fa0, 0xc0002f2f01) /home/aula/go/pkg/mod/github.com/lightninglabs/gozmq@v0.0.0-20180324010646-462a8a753885/zmq.go:378 +0x40 github.com/btcsuite/btcwallet/chain.(BitcoindConn).blockEventHandler(0xc00046a000, 0xc000252d50) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_conn.go:184 +0x17a created by github.com/btcsuite/btcwallet/chain.(BitcoindConn).Start /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_conn.go:139 +0x549

goroutine 53 [select]: github.com/btcsuite/btcwallet/wallet.(Wallet).handleChainNotifications(0xc000442000) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/wallet/chainntfns.go:88 +0x19d created by github.com/btcsuite/btcwallet/wallet.(Wallet).SynchronizeRPC /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/wallet/wallet.go:191 +0x153

goroutine 47 [select]: github.com/btcsuite/btcwallet/wallet.(Wallet).walletLocker(0xc000442000) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/wallet/wallet.go:1238 +0x2c5 created by github.com/btcsuite/btcwallet/wallet.(Wallet).Start /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/wallet/wallet.go:148 +0xef

goroutine 52 [select]: github.com/btcsuite/btcwallet/chain.(BitcoindClient).rescanHandler(0xc00012a7e0) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_client.go:515 +0x109 created by github.com/btcsuite/btcwallet/chain.(BitcoindClient).Start /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_client.go:476 +0x3ef

goroutine 98 [chan receive]: google.golang.org/grpc.(addrConn).resetTransport(0xc000444000) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/clientconn.go:1070 +0x9e5 created by google.golang.org/grpc.(addrConn).connect /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/clientconn.go:684 +0xb6

goroutine 85 [IO wait]: internal/poll.runtime_pollWait(0x7f01036ac980, 0x72, 0x0) /snap/go/3947/src/runtime/netpoll.go:182 +0x56 internal/poll.(pollDesc).wait(0xc0001b5798, 0x72, 0x0, 0x0, 0x11b1850) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:87 +0x9b internal/poll.(pollDesc).waitRead(...) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(FD).Accept(0xc0001b5780, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /snap/go/3947/src/internal/poll/fd_unix.go:384 +0x1ba net.(netFD).accept(0xc0001b5780, 0xc0000ac001, 0x0, 0x0) /snap/go/3947/src/net/fd_unix.go:238 +0x42 net.(TCPListener).accept(0xc0000ac448, 0xc000067e80, 0xc000067e88, 0x18) /snap/go/3947/src/net/tcpsock_posix.go:139 +0x32 net.(TCPListener).Accept(0xc0000ac448, 0x1330218, 0xc000135200, 0x14b3120, 0xc0000ac040) /snap/go/3947/src/net/tcpsock.go:260 +0x48 google.golang.org/grpc.(Server).Serve(0xc000135200, 0x14a14e0, 0xc0000ac448, 0x0, 0x0) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/server.go:556 +0x1e9 github.com/lightningnetwork/lnd.(rpcServer).Start.func2(0x14a14e0, 0xc0000ac448, 0xc0000b4780) /home/aula/go/src/github.com/lightningnetwork/lnd/rpcserver.go:636 +0xeb created by github.com/lightningnetwork/lnd.(*rpcServer).Start /home/aula/go/src/github.com/lightningnetwork/lnd/rpcserver.go:634 +0x2d0

goroutine 84 [select]: github.com/lightningnetwork/lnd.(channelNotifier).SubscribeChans.func1(0xc0003392c0, 0xc000294780, 0xc000706480) /home/aula/go/src/github.com/lightningnetwork/lnd/channel_notifier.go:68 +0x12c created by github.com/lightningnetwork/lnd.(channelNotifier).SubscribeChans /home/aula/go/src/github.com/lightningnetwork/lnd/channel_notifier.go:56 +0xd6

goroutine 87 [chan receive (nil chan)]: github.com/lightningnetwork/lnd/lnrpc.RegisterLightningHandlerFromEndpoint.func1.1(0x14a5b20, 0xc0000a8038, 0xc000298c00, 0xc0000a9120, 0xf) /home/aula/go/src/github.com/lightningnetwork/lnd/lnrpc/rpc.pb.gw.go:1021 +0x4c created by github.com/lightningnetwork/lnd/lnrpc.RegisterLightningHandlerFromEndpoint.func1 /home/aula/go/src/github.com/lightningnetwork/lnd/lnrpc/rpc.pb.gw.go:1020 +0x1b5

goroutine 88 [IO wait]: internal/poll.runtime_pollWait(0x7f01036ac8b0, 0x72, 0x0) /snap/go/3947/src/runtime/netpoll.go:182 +0x56 internal/poll.(pollDesc).wait(0xc0001b5818, 0x72, 0x0, 0x0, 0x11b1850) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:87 +0x9b internal/poll.(pollDesc).waitRead(...) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(FD).Accept(0xc0001b5800, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /snap/go/3947/src/internal/poll/fd_unix.go:384 +0x1ba net.(netFD).accept(0xc0001b5800, 0xc0001b1920, 0x30, 0x30) /snap/go/3947/src/net/fd_unix.go:238 +0x42 net.(TCPListener).accept(0xc0000ac478, 0xc0001b1920, 0x33, 0xc000317e70) /snap/go/3947/src/net/tcpsock_posix.go:139 +0x32 net.(TCPListener).Accept(0xc0000ac478, 0x13310b0, 0xc000317e78, 0xc000317e70, 0x40e0a8) /snap/go/3947/src/net/tcpsock.go:260 +0x48 crypto/tls.(listener).Accept(0xc00000f220, 0x10fd520, 0xc0001b1920, 0x10040e0, 0x1dbb390) /snap/go/3947/src/crypto/tls/tls.go:62 +0x37 net/http.(Server).Serve(0xc0002029c0, 0x149ce60, 0xc00000f220, 0x0, 0x0) /snap/go/3947/src/net/http/server.go:2859 +0x22d net/http.Serve(...) /snap/go/3947/src/net/http/server.go:2456 github.com/lightningnetwork/lnd.(rpcServer).Start.func4(0x149ce60, 0xc00000f220, 0xc0000d4f50) /home/aula/go/src/github.com/lightningnetwork/lnd/rpcserver.go:682 +0x119 created by github.com/lightningnetwork/lnd.(rpcServer).Start /home/aula/go/src/github.com/lightningnetwork/lnd/rpcserver.go:680 +0x4d8

goroutine 117 [select]: google.golang.org/grpc/internal/transport.(controlBuffer).get(0xc0006f2c00, 0x1, 0x0, 0x0, 0x0, 0x0) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/controlbuf.go:317 +0x104 google.golang.org/grpc/internal/transport.(loopyWriter).run(0xc000295a40, 0x0, 0x0) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/controlbuf.go:435 +0x1b6 google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc000306700) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/http2_client.go:328 +0x7b created by google.golang.org/grpc/internal/transport.newHTTP2Client /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/http2_client.go:326 +0xe6d

goroutine 206 [select]: github.com/lightningnetwork/lnd/lnwallet.(SigPool).poolWorker(0xc0002f21e0) /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:191 +0x17b created by github.com/lightningnetwork/lnd/lnwallet.(SigPool).Start /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:164 +0x79

goroutine 207 [select]: github.com/lightningnetwork/lnd/lnwallet.(SigPool).poolWorker(0xc0002f21e0) /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:191 +0x17b created by github.com/lightningnetwork/lnd/lnwallet.(SigPool).Start /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:164 +0x79

goroutine 116 [IO wait]: internal/poll.runtime_pollWait(0x7f01036ac4a0, 0x72, 0xffffffffffffffff) /snap/go/3947/src/runtime/netpoll.go:182 +0x56 internal/poll.(pollDesc).wait(0xc00019a318, 0x72, 0x600, 0x646, 0xffffffffffffffff) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:87 +0x9b internal/poll.(pollDesc).waitRead(...) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(FD).Read(0xc00019a300, 0xc000340000, 0x646, 0x646, 0x0, 0x0, 0x0) /snap/go/3947/src/internal/poll/fd_unix.go:169 +0x19b net.(netFD).Read(0xc00019a300, 0xc000340000, 0x646, 0x646, 0x203000, 0x0, 0x641) /snap/go/3947/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc000010028, 0xc000340000, 0x646, 0x646, 0x0, 0x0, 0x0) /snap/go/3947/src/net/net.go:177 +0x69 crypto/tls.(atLeastReader).Read(0xc000263120, 0xc000340000, 0x646, 0x646, 0x1e, 0x1486c60, 0xc000061a78) /snap/go/3947/src/crypto/tls/conn.go:761 +0x60 bytes.(Buffer).ReadFrom(0xc000308cd8, 0x1486a80, 0xc000263120, 0x40b925, 0x10340e0, 0x1168660) /snap/go/3947/src/bytes/buffer.go:207 +0xbd crypto/tls.(Conn).readFromUntil(0xc000308a80, 0x1487fc0, 0xc000010028, 0x5, 0xc000010028, 0x9) /snap/go/3947/src/crypto/tls/conn.go:783 +0xf8 crypto/tls.(Conn).readRecordOrCCS(0xc000308a80, 0x1331000, 0xc000308bb8, 0x0) /snap/go/3947/src/crypto/tls/conn.go:590 +0x125 crypto/tls.(Conn).readRecord(...) /snap/go/3947/src/crypto/tls/conn.go:558 crypto/tls.(Conn).Read(0xc000308a80, 0xc000280000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /snap/go/3947/src/crypto/tls/conn.go:1236 +0x137 bufio.(Reader).Read(0xc0002959e0, 0xc0002da038, 0x9, 0x9, 0xc000056e00, 0x7f01036f3008, 0x0) /snap/go/3947/src/bufio/bufio.go:223 +0x23e io.ReadAtLeast(0x14868e0, 0xc0002959e0, 0xc0002da038, 0x9, 0x9, 0x9, 0xc1bcd9, 0xc00031cbd0, 0xc000310004) /snap/go/3947/src/io/io.go:310 +0x88 io.ReadFull(...) /snap/go/3947/src/io/io.go:329 golang.org/x/net/http2.readFrameHeader(0xc0002da038, 0x9, 0x9, 0x14868e0, 0xc0002959e0, 0x0, 0x0, 0xc00031cbd0, 0x0) /home/aula/go/pkg/mod/golang.org/x/net@v0.0.0-20190206173232-65e2d4e15006/http2/frame.go:237 +0x88 golang.org/x/net/http2.(Framer).ReadFrame(0xc0002da000, 0xc00031cbd0, 0xc00031cb00, 0x0, 0x0) /home/aula/go/pkg/mod/golang.org/x/net@v0.0.0-20190206173232-65e2d4e15006/http2/frame.go:492 +0xa1 google.golang.org/grpc/internal/transport.(http2Client).reader(0xc000306700) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/http2_client.go:1221 +0x168 created by google.golang.org/grpc/internal/transport.newHTTP2Client /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/http2_client.go:286 +0xd15

goroutine 258 [select]: github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).RegisterBlockEpochNtfn.func1(0xc00093ac80, 0xc0000b43c0) /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:922 +0x137 created by github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).RegisterBlockEpochNtfn /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:918 +0x26f

goroutine 101 [select]: google.golang.org/grpc/internal/transport.(controlBuffer).get(0xc000078680, 0x1, 0x0, 0x0, 0x0, 0x0) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/controlbuf.go:317 +0x104 google.golang.org/grpc/internal/transport.(loopyWriter).run(0xc0001d6ae0, 0x0, 0x0) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/controlbuf.go:435 +0x1b6 google.golang.org/grpc/internal/transport.newHTTP2Server.func2(0xc00033d680) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/http2_server.go:276 +0xcb created by google.golang.org/grpc/internal/transport.newHTTP2Server /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/http2_server.go:273 +0xfba

goroutine 102 [select]: google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc00033d680) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/http2_server.go:921 +0x1ed created by google.golang.org/grpc/internal/transport.newHTTP2Server /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/http2_server.go:282 +0xfdf

goroutine 103 [IO wait]: internal/poll.runtime_pollWait(0x7f01036ac3d0, 0x72, 0xffffffffffffffff) /snap/go/3947/src/runtime/netpoll.go:182 +0x56 internal/poll.(pollDesc).wait(0xc00019a418, 0x72, 0x200, 0x205, 0xffffffffffffffff) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:87 +0x9b internal/poll.(pollDesc).waitRead(...) /snap/go/3947/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(FD).Read(0xc00019a400, 0xc0001f0000, 0x205, 0x205, 0x0, 0x0, 0x0) /snap/go/3947/src/internal/poll/fd_unix.go:169 +0x19b net.(netFD).Read(0xc00019a400, 0xc0001f0000, 0x205, 0x205, 0x203000, 0x0, 0x200) /snap/go/3947/src/net/fd_unix.go:202 +0x4f net.(conn).Read(0xc0000ac040, 0xc0001f0000, 0x205, 0x205, 0x0, 0x0, 0x0) /snap/go/3947/src/net/net.go:177 +0x69 crypto/tls.(atLeastReader).Read(0xc00000f840, 0xc0001f0000, 0x205, 0x205, 0x1e, 0x1486c60, 0xc00071ba08) /snap/go/3947/src/crypto/tls/conn.go:761 +0x60 bytes.(Buffer).ReadFrom(0xc0001bfe58, 0x1486a80, 0xc00000f840, 0x40b925, 0x10340e0, 0x1168660) /snap/go/3947/src/bytes/buffer.go:207 +0xbd crypto/tls.(Conn).readFromUntil(0xc0001bfc00, 0x1487fc0, 0xc0000ac040, 0x5, 0xc0000ac040, 0x9) /snap/go/3947/src/crypto/tls/conn.go:783 +0xf8 crypto/tls.(Conn).readRecordOrCCS(0xc0001bfc00, 0x1331000, 0xc0001bfd38, 0x0) /snap/go/3947/src/crypto/tls/conn.go:590 +0x125 crypto/tls.(Conn).readRecord(...) /snap/go/3947/src/crypto/tls/conn.go:558 crypto/tls.(Conn).Read(0xc0001bfc00, 0xc0006e4000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /snap/go/3947/src/crypto/tls/conn.go:1236 +0x137 bufio.(Reader).Read(0xc0001d6a80, 0xc00032e118, 0x9, 0x9, 0xc000056e00, 0x7f01036f36d0, 0x0) /snap/go/3947/src/bufio/bufio.go:223 +0x23e io.ReadAtLeast(0x14868e0, 0xc0001d6a80, 0xc00032e118, 0x9, 0x9, 0x9, 0xc1bcd9, 0xc0002dc000, 0xc0002d0004) /snap/go/3947/src/io/io.go:310 +0x88 io.ReadFull(...) /snap/go/3947/src/io/io.go:329 golang.org/x/net/http2.readFrameHeader(0xc00032e118, 0x9, 0x9, 0x14868e0, 0xc0001d6a80, 0x0, 0x0, 0xc0002dc000, 0x0) /home/aula/go/pkg/mod/golang.org/x/net@v0.0.0-20190206173232-65e2d4e15006/http2/frame.go:237 +0x88 golang.org/x/net/http2.(Framer).ReadFrame(0xc00032e0e0, 0xc0002dc000, 0xc0002dc000, 0x0, 0x0) /home/aula/go/pkg/mod/golang.org/x/net@v0.0.0-20190206173232-65e2d4e15006/http2/frame.go:492 +0xa1 google.golang.org/grpc/internal/transport.(http2Server).HandleStreams(0xc00033d680, 0xc00031cae0, 0x1330270) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/internal/transport/http2_server.go:429 +0x7c google.golang.org/grpc.(Server).serveStreams(0xc000135200, 0x14b44a0, 0xc00033d680) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/server.go:681 +0xdd google.golang.org/grpc.(Server).handleRawConn.func1(0xc000135200, 0x14b44a0, 0xc00033d680) /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/server.go:643 +0x43 created by google.golang.org/grpc.(*Server).handleRawConn /home/aula/go/pkg/mod/google.golang.org/grpc@v1.18.0/server.go:642 +0x583

goroutine 205 [select]: github.com/lightningnetwork/lnd/lnwallet.(SigPool).poolWorker(0xc0002f21e0) /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:191 +0x17b created by github.com/lightningnetwork/lnd/lnwallet.(SigPool).Start /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:164 +0x79

goroutine 259 [select]: github.com/lightningnetwork/lnd/htlcswitch.(DecayedLog).garbageCollector(0xc0002f2000, 0xc00048a0f0) /home/aula/go/src/github.com/lightningnetwork/lnd/htlcswitch/decayedlog.go:161 +0x118 created by github.com/lightningnetwork/lnd/htlcswitch.(DecayedLog).Start /home/aula/go/src/github.com/lightningnetwork/lnd/htlcswitch/decayedlog.go:113 +0x304

goroutine 203 [select]: github.com/lightningnetwork/lnd/lnwallet.(SigPool).poolWorker(0xc0002f21e0) /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:191 +0x17b created by github.com/lightningnetwork/lnd/lnwallet.(SigPool).Start /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:164 +0x79

goroutine 234 [select]: github.com/btcsuite/btcwallet/chain.(BitcoindClient).rescanHandler(0xc00012a360) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_client.go:515 +0x109 created by github.com/btcsuite/btcwallet/chain.(BitcoindClient).Start /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_client.go:476 +0x3ef

goroutine 202 [select]: github.com/lightningnetwork/lnd/lnwallet.(SigPool).poolWorker(0xc0002f21e0) /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:191 +0x17b created by github.com/lightningnetwork/lnd/lnwallet.(SigPool).Start /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:164 +0x79

goroutine 145 [select]: github.com/btcsuite/btcwallet/chain.(BitcoindClient).ntfnHandler(0xc00012a7e0) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_client.go:596 +0x157 created by github.com/btcsuite/btcwallet/chain.(BitcoindClient).NotifyBlocks /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_client.go:302 +0x362

goroutine 204 [select]: github.com/lightningnetwork/lnd/lnwallet.(SigPool).poolWorker(0xc0002f21e0) /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:191 +0x17b created by github.com/lightningnetwork/lnd/lnwallet.(SigPool).Start /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:164 +0x79

goroutine 260 [select]: github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start.func1(0xc00093acc0) /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:68 +0x2e8 created by github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:59 +0x6c

goroutine 261 [select]: github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).RegisterBlockEpochNtfn.func1(0xc00093ad00, 0xc0000b43c0) /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:922 +0x137 created by github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).RegisterBlockEpochNtfn /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:918 +0x26f

goroutine 208 [select]: github.com/lightningnetwork/lnd/lnwallet.(SigPool).poolWorker(0xc0002f21e0) /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:191 +0x17b created by github.com/lightningnetwork/lnd/lnwallet.(SigPool).Start /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:164 +0x79

goroutine 209 [select]: github.com/lightningnetwork/lnd/lnwallet.(SigPool).poolWorker(0xc0002f21e0) /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:191 +0x17b created by github.com/lightningnetwork/lnd/lnwallet.(SigPool).Start /home/aula/go/src/github.com/lightningnetwork/lnd/lnwallet/sigpool.go:164 +0x79

goroutine 226 [select]: github.com/lightningnetwork/lnd/pool.(Worker).requestHandler(0xc0002f20a0) /home/aula/go/src/github.com/lightningnetwork/lnd/pool/worker.go:153 +0xf4 created by github.com/lightningnetwork/lnd/pool.(Worker).Start.func1 /home/aula/go/src/github.com/lightningnetwork/lnd/pool/worker.go:99 +0x60

goroutine 227 [select]: github.com/lightningnetwork/lnd/pool.(Worker).requestHandler(0xc0002f2190) /home/aula/go/src/github.com/lightningnetwork/lnd/pool/worker.go:153 +0xf4 created by github.com/lightningnetwork/lnd/pool.(Worker).Start.func1 /home/aula/go/src/github.com/lightningnetwork/lnd/pool/worker.go:99 +0x60

goroutine 228 [select]: github.com/btcsuite/btcwallet/chain.(ConcurrentQueue).Start.func1(0xc0003383a0) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/queue.go:55 +0x2b6 created by github.com/btcsuite/btcwallet/chain.(ConcurrentQueue).Start /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/queue.go:47 +0x3f

goroutine 246 [select]: github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start.func1(0xc000078240) /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:68 +0x2e8 created by github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:59 +0x6c

goroutine 224 [select]: github.com/lightningnetwork/lnd/subscribe.(Server).subscriptionHandler(0xc000078840) /home/aula/go/src/github.com/lightningnetwork/lnd/subscribe/subscribe.go:167 +0x147 created by github.com/lightningnetwork/lnd/subscribe.(Server).Start /home/aula/go/src/github.com/lightningnetwork/lnd/subscribe/subscribe.go:91 +0x71

goroutine 240 [select]: github.com/btcsuite/btcwallet/chain.(BitcoindClient).ntfnHandler(0xc00012a360) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_client.go:596 +0x157 created by github.com/btcsuite/btcwallet/chain.(BitcoindClient).NotifyBlocks /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_client.go:302 +0x362

goroutine 225 [select]: github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start.func1(0xc00093ac40) /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:68 +0x2e8 created by github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:59 +0x6c

goroutine 223 [select]: github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).notificationDispatcher(0xc0000b43c0) /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:182 +0x13b created by github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).Start /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:138 +0x26c

goroutine 262 [select]: github.com/lightningnetwork/lnd/htlcswitch.(Switch).htlcForwarder(0xc0001acd00) /home/aula/go/src/github.com/lightningnetwork/lnd/htlcswitch/switch.go:1545 +0x378 created by github.com/lightningnetwork/lnd/htlcswitch.(Switch).Start /home/aula/go/src/github.com/lightningnetwork/lnd/htlcswitch/switch.go:1729 +0x107

goroutine 289 [select]: github.com/lightningnetwork/lnd/contractcourt.(ChannelArbitrator).channelAttendant(0xc00000c1e0, 0xc00008e7d2) /home/aula/go/src/github.com/lightningnetwork/lnd/contractcourt/channel_arbitrator.go:1964 +0x28e created by github.com/lightningnetwork/lnd/contractcourt.(ChannelArbitrator).Start /home/aula/go/src/github.com/lightningnetwork/lnd/contractcourt/channel_arbitrator.go:424 +0x645

goroutine 279 [select]: github.com/lightningnetwork/lnd.(utxoNursery).incubator(0xc0000b1b00, 0xc000033af0) /home/aula/go/src/github.com/lightningnetwork/lnd/utxonursery.go:716 +0x118 created by github.com/lightningnetwork/lnd.(utxoNursery).Start /home/aula/go/src/github.com/lightningnetwork/lnd/utxonursery.go:308 +0x2db

goroutine 249 [select]: github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).RegisterBlockEpochNtfn.func1(0xc0000b1fc0, 0xc0000b43c0) /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:922 +0x137 created by github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).RegisterBlockEpochNtfn /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:918 +0x26f

goroutine 248 [select]: github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start.func1(0xc0000b1f80) /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:68 +0x2e8 created by github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:59 +0x6c

goroutine 250 [select]: github.com/lightningnetwork/lnd/sweep.(UtxoSweeper).collector(0xc0001b4700, 0xc0001d7f20, 0xc00008e7d2, 0x0, 0x0) /home/aula/go/src/github.com/lightningnetwork/lnd/sweep/sweeper.go:452 +0x1d7 github.com/lightningnetwork/lnd/sweep.(UtxoSweeper).Start.func1(0xc000033750, 0xc0001b4700, 0x8e7d2) /home/aula/go/src/github.com/lightningnetwork/lnd/sweep/sweeper.go:350 +0x96 created by github.com/lightningnetwork/lnd/sweep.(*UtxoSweeper).Start /home/aula/go/src/github.com/lightningnetwork/lnd/sweep/sweeper.go:346 +0x534

goroutine 256 [select]: github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start.func1(0xc0006f2140) /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:68 +0x2e8 created by github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:59 +0x6c

goroutine 357 [select]: github.com/lightningnetwork/lnd/routing/chainview.(BitcoindFilteredChainView).chainFilterer(0xc00019ecb0) /home/aula/go/src/github.com/lightningnetwork/lnd/routing/chainview/bitcoind.go:297 +0x1bf created by github.com/lightningnetwork/lnd/routing/chainview.(BitcoindFilteredChainView).Start /home/aula/go/src/github.com/lightningnetwork/lnd/routing/chainview/bitcoind.go:112 +0x17f

goroutine 257 [select]: github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).RegisterBlockEpochNtfn.func1(0xc0006f2180, 0xc0000b43c0) /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:922 +0x137 created by github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).RegisterBlockEpochNtfn /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:918 +0x26f

goroutine 336 [select]: github.com/btcsuite/btcwallet/chain.(BitcoindClient).rescanHandler(0xc00012a480) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_client.go:515 +0x109 created by github.com/btcsuite/btcwallet/chain.(BitcoindClient).Start /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_client.go:476 +0x3ef

goroutine 280 [select]: github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start.func1(0xc0006f3200) /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:68 +0x2e8 created by github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:59 +0x6c

goroutine 281 [select]: github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).RegisterBlockEpochNtfn.func1(0xc0006f3240, 0xc0000b43c0) /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:922 +0x137 created by github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).RegisterBlockEpochNtfn /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:918 +0x26f

goroutine 330 [select]: github.com/btcsuite/btcwallet/chain.(ConcurrentQueue).Start.func1(0xc0003383c0) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/queue.go:55 +0x2b6 created by github.com/btcsuite/btcwallet/chain.(ConcurrentQueue).Start /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/queue.go:47 +0x3f

goroutine 306 [select]: github.com/lightningnetwork/lnd.(breachArbiter).contractObserver(0xc0000b1b40) /home/aula/go/src/github.com/lightningnetwork/lnd/breacharbiter.go:261 +0x182 created by github.com/lightningnetwork/lnd.(breachArbiter).start /home/aula/go/src/github.com/lightningnetwork/lnd/breacharbiter.go:223 +0x736

goroutine 324 [select]: github.com/lightningnetwork/lnd/routing.(ChannelRouter).UpdateEdge(0xc0000b46e0, 0xc000758e10, 0xc0000c3600, 0x0) /home/aula/go/src/github.com/lightningnetwork/lnd/routing/router.go:2171 +0x133 github.com/lightningnetwork/lnd/discovery.(AuthenticatedGossiper).updateChannel(0xc0001aa160, 0xc00076cdc0, 0xc000758e10, 0x0, 0x44d316, 0xc0000b1700, 0x8) /home/aula/go/src/github.com/lightningnetwork/lnd/discovery/gossiper.go:2573 +0x52f github.com/lightningnetwork/lnd/discovery.(AuthenticatedGossiper).retransmitStaleChannels(0xc0001aa160, 0x1331278, 0xc0002f3680) /home/aula/go/src/github.com/lightningnetwork/lnd/discovery/gossiper.go:1293 +0x232 github.com/lightningnetwork/lnd/discovery.(AuthenticatedGossiper).networkHandler(0xc0001aa160) /home/aula/go/src/github.com/lightningnetwork/lnd/discovery/gossiper.go:976 +0x13a created by github.com/lightningnetwork/lnd/discovery.(*AuthenticatedGossiper).Start /home/aula/go/src/github.com/lightningnetwork/lnd/discovery/gossiper.go:500 +0x1ae

goroutine 300 [select]: github.com/lightningnetwork/lnd/contractcourt.(chainWatcher).closeObserver(0xc000702380, 0xc00000fb60) /home/aula/go/src/github.com/lightningnetwork/lnd/contractcourt/chain_watcher.go:407 +0x1bf created by github.com/lightningnetwork/lnd/contractcourt.(chainWatcher).Start /home/aula/go/src/github.com/lightningnetwork/lnd/contractcourt/chain_watcher.go:275 +0x2dc

goroutine 307 [select]: github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start.func1(0xc0006f3800) /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:68 +0x2e8 created by github.com/lightningnetwork/lnd/queue.(ConcurrentQueue).Start /home/aula/go/src/github.com/lightningnetwork/lnd/queue/queue.go:59 +0x6c

goroutine 308 [select]: github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).RegisterBlockEpochNtfn.func1(0xc0006f3840, 0xc0000b43c0) /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:922 +0x137 created by github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify.(BitcoindNotifier).RegisterBlockEpochNtfn /home/aula/go/src/github.com/lightningnetwork/lnd/chainntnfs/bitcoindnotify/bitcoind.go:918 +0x26f

goroutine 323 [select]: github.com/lightningnetwork/lnd/discovery.(SyncManager).syncerHandler(0xc0001daa80) /home/aula/go/src/github.com/lightningnetwork/lnd/discovery/sync_manager.go:211 +0x2aa created by github.com/lightningnetwork/lnd/discovery.(SyncManager).Start.func1 /home/aula/go/src/github.com/lightningnetwork/lnd/discovery/sync_manager.go:149 +0x63

goroutine 322 [select]: github.com/lightningnetwork/lnd/discovery.(reliableSender).peerHandler(0xc00019e230, 0xc00070d020, 0xc00070d080, 0x1edc93d6b86fad02, 0xf7f5adefedbc6945, 0xc0fdc27e01a932a, 0xa0b9f3c6c1344b54, 0xfe8b2b) /home/aula/go/src/github.com/lightningnetwork/lnd/discovery/reliable_sender.go:174 +0x1c3 created by github.com/lightningnetwork/lnd/discovery.(reliableSender).spawnPeerHandler /home/aula/go/src/github.com/lightningnetwork/lnd/discovery/reliable_sender.go:147 +0x26a

goroutine 342 [select]: github.com/btcsuite/btcwallet/chain.(BitcoindClient).ntfnHandler(0xc00012a480) /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_client.go:596 +0x157 created by github.com/btcsuite/btcwallet/chain.(BitcoindClient).NotifyBlocks /home/aula/go/pkg/mod/github.com/btcsuite/btcwallet@v0.0.0-20190628225330-4a9774585e57/chain/bitcoind_client.go:302 +0x362

goroutine 356 [sync.Cond.Wait]: runtime.goparkunlock(...) /snap/go/3947/src/runtime/proc.go:307 sync.runtime_notifyListWait(0xc0000b0ad0, 0xc000000000) /snap/go/3947/src/runtime/sema.go:510 +0xf9 sync.(Cond).Wait(0xc0000b0ac0) /snap/go/3947/src/sync/cond.go:56 +0x9e github.com/lightningnetwork/lnd/routing/chainview.(blockEventQueue).queueCoordinator(0xc0000d4640) /home/aula/go/src/github.com/lightningnetwork/lnd/routing/chainview/queue.go:89 +0x86 created by github.com/lightningnetwork/lnd/routing/chainview.(*blockEventQueue).Start /home/aula/go/src/github.com/lightningnetwork/lnd/routing/chainview/queue.go:66 +0x5c

wpaulino commented 5 years ago

Looks similar to #2796. Has the node experienced any power outages or anything else that could lead to database corruption?

tomasvanagas commented 5 years ago

Yeah it went down a couple times. Thanks for fast response, I'll take a look

wpaulino commented 5 years ago

In any case, I'd suggest taking a look at our recovery documentation.

halseth commented 5 years ago

Closing as duplicate of #2796