ethereum / go-ethereum

Go implementation of the Ethereum protocol
https://geth.ethereum.org
GNU Lesser General Public License v3.0
47.54k stars 20.13k forks source link

Default Windows 1.8.22 install fails every few minutes with Out Of Memory Exception #19031

Closed mmcc1 closed 5 years ago

mmcc1 commented 5 years ago

When attempting to sync for the first time with the rinkeby network, 1.8.22-stable keeps crashing every 3-5 minutes with the below error.

This is on a machine with 32GB RAM and 60-80% resources free.

System information

Geth version: geth version

Geth Version: 1.8.22-stable Git Commit: 7fa3509e2eaf1a4ebc12344590e5699406690f15 Architecture: 386 Protocol Versions: [63 62] Network Id: 1 Go Version: go1.11.5 Operating System: windows GOPATH= GOROOT=C:\go

Expected behaviour

Should sync without crashing

Actual behaviour

Crashes every 5-10 minutes. Seems like a memory leak.

Steps to reproduce the behaviour

Default install, run the following command:

geth --rinkeby --rpc --rpcapi="personal,eth,network,web3,net" --ipcpath "~/AppData/Roaming/Ethereum/geth.ipc" --syncmode "fast" --cache 2048

Backtrace

runtime: out of memory: cannot allocate 268435456-byte block (1420722176 in use)
fatal error: out of memory

runtime stack:
runtime.throw(0x1130a4c, 0xd)
        C:/go/src/runtime/panic.go:608 +0x64
runtime.largeAlloc(0x10000000, 0x450101, 0x2d90358)
        C:/go/src/runtime/malloc.go:1021 +0x107
runtime.mallocgc.func1()
        C:/go/src/runtime/malloc.go:914 +0x39
runtime.systemstack(0x3773468)
        C:/go/src/runtime/asm_386.s:396 +0x53
runtime.mstart()
        C:/go/src/runtime/proc.go:1229

goroutine 264 [running]:
runtime.systemstack_switch()
        C:/go/src/runtime/asm_386.s:357 fp=0x4797145c sp=0x47971458 pc=0x452aa0
runtime.mallocgc(0x10000000, 0x101dfa0, 0x1, 0x299a0280)
        C:/go/src/runtime/malloc.go:913 +0x676 fp=0x479714b0 sp=0x4797145c pc=0x40b7f6
runtime.makeslice(0x101dfa0, 0x0, 0x10000000, 0x3644000, 0x31d58c00, 0x5dcd01)
        C:/go/src/runtime/slice.go:70 +0x59 fp=0x479714c4 sp=0x479714b0 pc=0x43e569
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/memdb.New(0x138d7b0, 0x13d08068, 0x10000000, 0x13fa47a0)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/memdb/memdb.go:470 +0xce fp=0x4797150c sp=0x479714c4 pc=0x5ea5ae
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).mpoolGet(0x13d6e120, 0x1afff, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_state.go:90 +0x92 fp=0x47971528 sp=0x4797150c pc=0x60d5b2
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).newMem(0x13d6e120, 0x1afff, 0x0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_state.go:147 +0x1a4 fp=0x47971574 sp=0x47971528 pc=0x60d994
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).rotateMem(0x13d6e120, 0x1afff, 0x0, 0x47971654, 0x0, 0x1f)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_write.go:46 +0x6a fp=0x4797159c sp=0x47971574 pc=0x61053a
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).flush.func1(0x2ee2e00)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_write.go:106 +0x23d fp=0x479715e0 sp=0x4797159c pc=0x621fcd
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).flush(0x13d6e120, 0x1afff, 0x2549edb0, 0x184bf, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_write.go:118 +0xe0 fp=0x4797164c sp=0x479715e0 pc=0x610720
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).writeLocked(0x13d6e120, 0x2a160720, 0x0, 0x1, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_write.go:157 +0x53 fp=0x47971710 sp=0x4797164c pc=0x6109c3
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).Write(0x13d6e120, 0x2a160720, 0x0, 0x500a9c00, 0x36f58000)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_write.go:317 +0x236 fp=0x4797183c sp=0x47971710 pc=0x611326
github.com/ethereum/go-ethereum/ethdb.(*ldbBatch).Write(0x4fbe7990, 0x1a6a7, 0x51b41400)
        C:/gopath/src/github.com/ethereum/go-ethereum/ethdb/database.go:374 +0x32 fp=0x47971854 sp=0x4797183c pc=0x628cd2
github.com/ethereum/go-ethereum/core.(*BlockChain).InsertReceiptChain(0x24d68000, 0x52b0f680, 0x8b, 0x8b, 0x595a6000, 0x8b, 0x8b, 0x0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/core/blockchain.go:869 +0xfe2 fp=0x47971d28 sp=0x47971854 pc=0x7b4982
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).commitFastSyncData(0x13c8c000, 0x66ab4000, 0x8b, 0x100, 0x28327680, 0x0, 0x66ab4000)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1641 +0x479 fp=0x47971ec4 sp=0x47971d28 pc=0x952439
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processFastSyncContent(0x13c8c000, 0x2ed4a000, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1560 +0x34f fp=0x47971fb4 sp=0x47971ec4 pc=0x95162f
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func7(0x4, 0x11bbc3c)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:477 +0x27 fp=0x47971fc8 sp=0x47971fb4 pc=0x9604b7
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1(0x13c8c000, 0x28327480, 0x31a3e0b0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:491 +0x46 fp=0x47971fe0 sp=0x47971fc8 pc=0x960526
runtime.goexit()
        C:/go/src/runtime/asm_386.s:1324 +0x1 fp=0x47971fe4 sp=0x47971fe0 pc=0x454421
created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:491 +0x8c

goroutine 1 [chan receive, 11 minutes]:
github.com/ethereum/go-ethereum/node.(*Node).Wait(0x13d56140)
        C:/gopath/src/github.com/ethereum/go-ethereum/node/node.go:464 +0x65
main.geth(0x13d8c160, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/cmd/geth/main.go:269 +0xc9
github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli%2ev1.HandleAction(0x1003e40, 0x11bb730, 0x13d8c160, 0x13d8c160, 0x13cedfb4)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli.v1/app.go:490 +0xa2
github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli%2ev1.(*App).Run(0x13cf7260, 0x13c70000, 0xa, 0x10, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli.v1/app.go:264 +0x4c5
main.main()
        C:/gopath/src/github.com/ethereum/go-ethereum/cmd/geth/main.go:254 +0x45

goroutine 6 [syscall, 11 minutes]:
os/signal.signal_recv(0x0)
        C:/go/src/runtime/sigqueue.go:139 +0x12f
os/signal.loop()
        C:/go/src/os/signal/signal_unix.go:23 +0x1a
created by os/signal.init.0
        C:/go/src/os/signal/signal_unix.go:29 +0x37

goroutine 7 [chan receive, 11 minutes]:
github.com/ethereum/go-ethereum/core.(*txSenderCacher).cache(0x13c06b58)
        C:/gopath/src/github.com/ethereum/go-ethereum/core/tx_cacher.go:63 +0x79
created by github.com/ethereum/go-ethereum/core.newTxSenderCacher
        C:/gopath/src/github.com/ethereum/go-ethereum/core/tx_cacher.go:55 +0x86

goroutine 8 [chan receive, 11 minutes]:
github.com/ethereum/go-ethereum/core.(*txSenderCacher).cache(0x13c06b58)
        C:/gopath/src/github.com/ethereum/go-ethereum/core/tx_cacher.go:63 +0x79
created by github.com/ethereum/go-ethereum/core.newTxSenderCacher
        C:/gopath/src/github.com/ethereum/go-ethereum/core/tx_cacher.go:55 +0x86

goroutine 9 [chan receive, 11 minutes]:
github.com/ethereum/go-ethereum/core.(*txSenderCacher).cache(0x13c06b58)
        C:/gopath/src/github.com/ethereum/go-ethereum/core/tx_cacher.go:63 +0x79
created by github.com/ethereum/go-ethereum/core.newTxSenderCacher
        C:/gopath/src/github.com/ethereum/go-ethereum/core/tx_cacher.go:55 +0x86

goroutine 10 [chan receive, 11 minutes]:
github.com/ethereum/go-ethereum/core.(*txSenderCacher).cache(0x13c06b58)
        C:/gopath/src/github.com/ethereum/go-ethereum/core/tx_cacher.go:63 +0x79
created by github.com/ethereum/go-ethereum/core.newTxSenderCacher
        C:/gopath/src/github.com/ethereum/go-ethereum/core/tx_cacher.go:55 +0x86

goroutine 13 [select, 11 minutes]:
github.com/ethereum/go-ethereum/core.(*ChainIndexer).updateLoop(0x13d40180)
        C:/gopath/src/github.com/ethereum/go-ethereum/core/chain_indexer.go:307 +0xe8
created by github.com/ethereum/go-ethereum/core.NewChainIndexer
        C:/gopath/src/github.com/ethereum/go-ethereum/core/chain_indexer.go:116 +0x20b

goroutine 36 [chan receive, 11 minutes]:
github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify.(*recursiveTree).dispatch(0x13d734c0)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/tree_recursive.go:125 +0xa9
created by github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify.newRecursiveTree
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/tree_recursive.go:119 +0x114

goroutine 35 [select]:
github.com/ethereum/go-ethereum/consensus/ethash.(*Ethash).remote(0x13d2c100, 0x0, 0x0, 0x0, 0x100)
        C:/gopath/src/github.com/ethereum/go-ethereum/consensus/ethash/sealer.go:307 +0x453
created by github.com/ethereum/go-ethereum/consensus/ethash.New
        C:/gopath/src/github.com/ethereum/go-ethereum/consensus/ethash/ethash.go:496 +0x285

goroutine 34 [chan receive]:
github.com/ethereum/go-ethereum/metrics.(*meterArbiter).tick(0x19cdba0)
        C:/gopath/src/github.com/ethereum/go-ethereum/metrics/meter.go:289 +0x5a
created by github.com/ethereum/go-ethereum/metrics.NewMeterForced
        C:/gopath/src/github.com/ethereum/go-ethereum/metrics/meter.go:70 +0xdb

goroutine 42 [select]:
github.com/ethereum/go-ethereum/accounts/keystore.(*KeyStore).updater(0x13d3c870)
        C:/gopath/src/github.com/ethereum/go-ethereum/accounts/keystore/keystore.go:203 +0xa3
created by github.com/ethereum/go-ethereum/accounts/keystore.(*KeyStore).Subscribe
        C:/gopath/src/github.com/ethereum/go-ethereum/accounts/keystore/keystore.go:190 +0xf0

goroutine 43 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
        C:/go/src/runtime/time.go:105 +0x14c
github.com/ethereum/go-ethereum/accounts/usbwallet.(*Hub).updater(0x13d40000)
        C:/gopath/src/github.com/ethereum/go-ethereum/accounts/usbwallet/hub.go:224 +0x37
created by github.com/ethereum/go-ethereum/accounts/usbwallet.(*Hub).Subscribe
        C:/gopath/src/github.com/ethereum/go-ethereum/accounts/usbwallet/hub.go:213 +0xfc

goroutine 44 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
        C:/go/src/runtime/time.go:105 +0x14c
github.com/ethereum/go-ethereum/accounts/usbwallet.(*Hub).updater(0x13d400c0)
        C:/gopath/src/github.com/ethereum/go-ethereum/accounts/usbwallet/hub.go:224 +0x37
created by github.com/ethereum/go-ethereum/accounts/usbwallet.(*Hub).Subscribe
        C:/gopath/src/github.com/ethereum/go-ethereum/accounts/usbwallet/hub.go:213 +0xfc

goroutine 45 [select, 11 minutes]:
github.com/ethereum/go-ethereum/accounts.(*Manager).update(0x13d2c500)
        C:/gopath/src/github.com/ethereum/go-ethereum/accounts/manager.go:95 +0x13a
created by github.com/ethereum/go-ethereum/accounts.NewManager
        C:/gopath/src/github.com/ethereum/go-ethereum/accounts/manager.go:68 +0x3cf

goroutine 46 [select]:
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0x13d8c420)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:206 +0xe1
created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util.NewBufferPool
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:237 +0x11e

goroutine 47 [select, 9 minutes]:
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0x13d6e120)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:90 +0xa1
created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:142 +0x338

goroutine 48 [select]:
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0x13d6e120)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_state.go:101 +0xc0
created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:143 +0x356

goroutine 49 [select, 9 minutes]:
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0x13d6e120)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:825 +0x1cd
created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:149 +0x480

goroutine 50 [select]:
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0x13d6e120)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:762 +0x10a
created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:150 +0x49e

goroutine 51 [select]:
github.com/ethereum/go-ethereum/ethdb.(*LDBDatabase).meter(0x25ad0d80, 0xb2d05e00, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/ethdb/database.go:337 +0x13c4
created by github.com/ethereum/go-ethereum/ethdb.(*LDBDatabase).Meter
        C:/gopath/src/github.com/ethereum/go-ethereum/ethdb/database.go:174 +0x3ab

goroutine 24 [select]:
github.com/ethereum/go-ethereum/core.(*BlockChain).update(0x24d68000)
        C:/gopath/src/github.com/ethereum/go-ethereum/core/blockchain.go:1529 +0xbe
created by github.com/ethereum/go-ethereum/core.NewBlockChain
        C:/gopath/src/github.com/ethereum/go-ethereum/core/blockchain.go:205 +0x7be

goroutine 25 [select, 11 minutes]:
github.com/ethereum/go-ethereum/core.(*ChainIndexer).eventLoop(0x13d40180, 0x13d51200, 0x13e80b80, 0x138ff30, 0x13d0f180)
        C:/gopath/src/github.com/ethereum/go-ethereum/core/chain_indexer.go:207 +0x1d8
created by github.com/ethereum/go-ethereum/core.(*ChainIndexer).Start
        C:/gopath/src/github.com/ethereum/go-ethereum/core/chain_indexer.go:148 +0xa9

goroutine 14 [select]:
github.com/ethereum/go-ethereum/core.(*TxPool).loop(0x13d6e240)
        C:/gopath/src/github.com/ethereum/go-ethereum/core/tx_pool.go:306 +0x28e
created by github.com/ethereum/go-ethereum/core.NewTxPool
        C:/gopath/src/github.com/ethereum/go-ethereum/core/tx_pool.go:278 +0x56b

goroutine 15 [select]:
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).qosTuner(0x13c8c000)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1721 +0x348
created by github.com/ethereum/go-ethereum/eth/downloader.New
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:238 +0x32c

goroutine 16 [select]:
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).runStateSync(0x13c8c000, 0x28327680, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/statesync.go:125 +0x2f3
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).stateFetcher(0x13c8c000)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/statesync.go:79 +0x2b
created by github.com/ethereum/go-ethereum/eth/downloader.New
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:239 +0x34a

goroutine 66 [select, 11 minutes]:
github.com/ethereum/go-ethereum/miner.(*worker).mainLoop(0x13c8c120)
        C:/gopath/src/github.com/ethereum/go-ethereum/miner/worker.go:408 +0x2b7
created by github.com/ethereum/go-ethereum/miner.newWorker
        C:/gopath/src/github.com/ethereum/go-ethereum/miner/worker.go:218 +0x458

goroutine 67 [select, 11 minutes]:
github.com/ethereum/go-ethereum/miner.(*worker).newWorkLoop(0x13c8c120, 0xb2d05e00, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/miner/worker.go:343 +0x253
created by github.com/ethereum/go-ethereum/miner.newWorker
        C:/gopath/src/github.com/ethereum/go-ethereum/miner/worker.go:219 +0x48c

goroutine 68 [select, 11 minutes]:
github.com/ethereum/go-ethereum/miner.(*worker).resultLoop(0x13c8c120)
        C:/gopath/src/github.com/ethereum/go-ethereum/miner/worker.go:542 +0xd2
created by github.com/ethereum/go-ethereum/miner.newWorker
        C:/gopath/src/github.com/ethereum/go-ethereum/miner/worker.go:220 +0x4aa

goroutine 69 [select, 11 minutes]:
github.com/ethereum/go-ethereum/miner.(*worker).taskLoop(0x13c8c120)
        C:/gopath/src/github.com/ethereum/go-ethereum/miner/worker.go:507 +0xcb
created by github.com/ethereum/go-ethereum/miner.newWorker
        C:/gopath/src/github.com/ethereum/go-ethereum/miner/worker.go:221 +0x4c8

goroutine 70 [select, 11 minutes]:
github.com/ethereum/go-ethereum/miner.(*Miner).update(0x13d20c40)
        C:/gopath/src/github.com/ethereum/go-ethereum/miner/miner.go:78 +0x214
created by github.com/ethereum/go-ethereum/miner.New
        C:/gopath/src/github.com/ethereum/go-ethereum/miner/miner.go:64 +0x12a

goroutine 71 [select]:
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0x25d620b0)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:206 +0xe1
created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util.NewBufferPool
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:237 +0x11e

goroutine 72 [select, 11 minutes]:
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0x13c8c480)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:90 +0xa1
created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:142 +0x338

goroutine 73 [select]:
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0x13c8c480)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_state.go:101 +0xc0
created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:143 +0x356

goroutine 74 [select, 11 minutes]:
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0x13c8c480)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:825 +0x1cd
created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:149 +0x480

goroutine 75 [select, 11 minutes]:
github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0x13c8c480)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:762 +0x10a
created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:150 +0x49e

goroutine 163 [select]:
github.com/ethereum/go-ethereum/p2p.(*Peer).run(0x13fb4300, 0xfbd6a0, 0x13e5aa40, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:215 +0x223
github.com/ethereum/go-ethereum/p2p.(*Server).runPeer(0x13d8e6e0, 0x13fb4300)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:1002 +0x107
created by github.com/ethereum/go-ethereum/p2p.(*Server).run
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:734 +0x11ae

goroutine 77 [IO wait, 11 minutes]:
internal/poll.runtime_pollWait(0x3714e58, 0x72, 0x138f170)
        C:/go/src/runtime/netpoll.go:173 +0x4b
internal/poll.(*pollDesc).wait(0x13fa83fc, 0x72, 0xbd4200, 0x0, 0x0)
        C:/go/src/internal/poll/fd_poll_runtime.go:85 +0x7d
internal/poll.(*ioSrv).ExecIO(0x19caad8, 0x13fa8314, 0x2549e160, 0x17, 0x1, 0x0)
        C:/go/src/internal/poll/fd_windows.go:222 +0xf1
internal/poll.(*FD).acceptOne(0x13fa8300, 0x37c, 0x2691a000, 0x2, 0x2, 0x13fa8314, 0x40b3fa, 0x13e8a120, 0x40ba2c, 0x8)
        C:/go/src/internal/poll/fd_windows.go:796 +0x82
internal/poll.(*FD).Accept(0x13fa8300, 0x13fc80e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        C:/go/src/internal/poll/fd_windows.go:830 +0x119
net.(*netFD).accept(0x13fa8300, 0x19d04e0, 0x268f21b0, 0x268f21b0)
        C:/go/src/net/fd_windows.go:193 +0x60
net.(*TCPListener).accept(0x13cfefc8, 0x405c2c, 0x268f2180, 0x0)
        C:/go/src/net/tcpsock_posix.go:139 +0x23
net.(*TCPListener).Accept(0x13cfefc8, 0x0, 0x268f2180, 0x2549e140, 0x2)
        C:/go/src/net/tcpsock.go:260 +0x3c
github.com/ethereum/go-ethereum/p2p.(*Server).listenLoop(0x13d8e6e0)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:845 +0x1b4
created by github.com/ethereum/go-ethereum/p2p.(*Server).setupListening
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:584 +0x149

goroutine 78 [select, 11 minutes]:
github.com/ethereum/go-ethereum/p2p/nat.Map(0x1394560, 0x13cf90e0, 0x25470700, 0x11265d8, 0x3, 0x765f, 0x765f, 0x112f855, 0xc)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/nat/nat.go:114 +0x37c
github.com/ethereum/go-ethereum/p2p.(*Server).setupListening.func1(0x13d8e6e0, 0x16b899c0)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:590 +0x68
created by github.com/ethereum/go-ethereum/p2p.(*Server).setupListening
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:589 +0x1c7

goroutine 79 [select, 11 minutes]:
github.com/ethereum/go-ethereum/p2p/nat.Map(0x1394560, 0x13cf90e0, 0x25470700, 0x11265e4, 0x3, 0x765f, 0x765f, 0x1135ae9, 0x12)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/nat/nat.go:114 +0x37c
created by github.com/ethereum/go-ethereum/p2p.(*Server).setupDiscovery
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:527 +0x498

goroutine 80 [select, 11 minutes]:
github.com/ethereum/go-ethereum/p2p/enode.(*DB).expirer(0x16b89880)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/enode/nodedb.go:300 +0xb7
created by github.com/ethereum/go-ethereum/p2p/enode.(*DB).ensureExpirer.func1
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/enode/nodedb.go:291 +0x32

goroutine 81 [select]:
github.com/ethereum/go-ethereum/p2p/discover.(*Table).loop(0x25672280)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:365 +0x27a
created by github.com/ethereum/go-ethereum/p2p/discover.newTable
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:126 +0x260

goroutine 82 [select]:
github.com/ethereum/go-ethereum/p2p/discover.(*udp).loop(0x13d21340)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/udp.go:436 +0x440
created by github.com/ethereum/go-ethereum/p2p/discover.newUDP
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/udp.go:278 +0x187

goroutine 83 [IO wait]:
internal/poll.runtime_pollWait(0x3714dd8, 0x72, 0x138f170)
        C:/go/src/runtime/netpoll.go:173 +0x4b
internal/poll.(*pollDesc).wait(0x13fa857c, 0x72, 0xbd4200, 0x0, 0x0)
        C:/go/src/internal/poll/fd_poll_runtime.go:85 +0x7d
internal/poll.(*ioSrv).ExecIO(0x19caad8, 0x13fa8494, 0x11bb6a8, 0xcb68b523, 0xb8877314, 0xa4708ead)
        C:/go/src/internal/poll/fd_windows.go:222 +0xf1
internal/poll.(*FD).ReadFrom(0x13fa8480, 0x13dc2000, 0x500, 0x500, 0x0, 0x0, 0x0, 0x0, 0x0)
        C:/go/src/internal/poll/fd_windows.go:620 +0xd0
net.(*netFD).readFrom(0x13fa8480, 0x13dc2000, 0x500, 0x500, 0x486bb8ec, 0x2daf4a20, 0x13dc2000, 0x0, 0x0)
        C:/go/src/net/fd_windows.go:158 +0x3f
net.(*UDPConn).readFrom(0x13cff008, 0x13dc2000, 0x500, 0x500, 0x101dce0, 0x1388d30, 0x10d6c60, 0x2a1fc1a0)
        C:/go/src/net/udpsock_posix.go:47 +0x3f
net.(*UDPConn).ReadFromUDP(0x13cff008, 0x13dc2000, 0x500, 0x500, 0x500, 0x0, 0x0, 0x0)
        C:/go/src/net/udpsock.go:109 +0x56
github.com/ethereum/go-ethereum/p2p/discover.(*udp).readLoop(0x13d21340, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/udp.go:570 +0xb1
created by github.com/ethereum/go-ethereum/p2p/discover.newUDP
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/udp.go:279 +0x1ad

goroutine 84 [select]:
github.com/ethereum/go-ethereum/p2p.(*Server).run(0x13d8e6e0, 0x13944e0, 0x26a96600)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:657 +0x641
created by github.com/ethereum/go-ethereum/p2p.(*Server).Start
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:461 +0x476

goroutine 85 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 86 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 87 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 88 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 89 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 90 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 91 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 92 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 93 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 94 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 95 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 96 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 97 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 98 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 99 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 100 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0x25b78fc0, 0x1000, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:56 +0x323
created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/bloombits.go:54 +0x4b

goroutine 5955 [select]:
github.com/ethereum/go-ethereum/p2p.(*Peer).run(0x4d05a7c0, 0xfbd6a0, 0x4d05a8c0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:215 +0x223
github.com/ethereum/go-ethereum/p2p.(*Server).runPeer(0x13d8e6e0, 0x4d05a7c0)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:1002 +0x107
created by github.com/ethereum/go-ethereum/p2p.(*Server).run
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:734 +0x11ae

goroutine 101 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*ProtocolManager).txBroadcastLoop(0x13d12000)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/handler.go:783 +0xd5
created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).Start
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/handler.go:215 +0x9f

goroutine 102 [chan receive, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*ProtocolManager).minedBroadcastLoop(0x13d12000)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/handler.go:773 +0x45
created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).Start
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/handler.go:219 +0x117

goroutine 103 [select]:
github.com/ethereum/go-ethereum/eth.(*ProtocolManager).syncer(0x13d12000)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/sync.go:145 +0x124
created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).Start
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/handler.go:222 +0x135

goroutine 104 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*ProtocolManager).txsyncLoop(0x13d12000)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/sync.go:109 +0x1c3
created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).Start
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/handler.go:223 +0x153

goroutine 105 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth/downloader.(*PublicDownloaderAPI).eventLoop(0x26b24110)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/api.go:63 +0x224
created by github.com/ethereum/go-ethereum/eth/downloader.NewPublicDownloaderAPI
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/api.go:49 +0xaa

goroutine 106 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth/filters.(*EventSystem).eventLoop(0x26b1d9a0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/filters/filter_system.go:464 +0x3c3
created by github.com/ethereum/go-ethereum/eth/filters.NewEventSystem
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/filters/filter_system.go:146 +0x319

goroutine 107 [chan receive, 1 minutes]:
github.com/ethereum/go-ethereum/eth/filters.(*PublicFilterAPI).timeoutLoop(0x26a313e0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/filters/api.go:83 +0xe2
created by github.com/ethereum/go-ethereum/eth/filters.NewPublicFilterAPI
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/filters/api.go:73 +0x112

goroutine 108 [syscall, 11 minutes, locked to thread]:
syscall.Syscall(0x74b2f750, 0x2, 0x3b8, 0xffffffff, 0x0, 0x0, 0x0, 0x0)
        C:/go/src/runtime/syscall_windows.go:172 +0xcf
syscall.WaitForSingleObject(0x3b8, 0xffffffff, 0x42a1f9, 0x4, 0x26b7c780)
        C:/go/src/syscall/zsyscall_windows.go:737 +0x53
github.com/ethereum/go-ethereum/vendor/gopkg.in/natefinch/npipe%2ev2.waitForCompletion(0x398, 0x26b9c580, 0x26b9f0a0, 0x25b62d5c, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/gopkg.in/natefinch/npipe.v2/npipe_windows.go:182 +0x2c
github.com/ethereum/go-ethereum/vendor/gopkg.in/natefinch/npipe%2ev2.(*PipeListener).AcceptPipe(0x26b9f0a0, 0x0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/gopkg.in/natefinch/npipe.v2/npipe_windows.go:327 +0x222
github.com/ethereum/go-ethereum/vendor/gopkg.in/natefinch/npipe%2ev2.(*PipeListener).Accept(0x26b9f0a0, 0x0, 0x0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/vendor/gopkg.in/natefinch/npipe.v2/npipe_windows.go:263 +0x25
github.com/ethereum/go-ethereum/rpc.(*Server).ServeListener(0x26b21fc0, 0x1392f10, 0x26b9f0a0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/ipc.go:30 +0x29
created by github.com/ethereum/go-ethereum/rpc.StartIPCEndpoint
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/endpoints.go:100 +0x1d4

goroutine 29 [select]:
github.com/ethereum/go-ethereum/eth/fetcher.(*Fetcher).loop(0x13d120e0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/fetcher/fetcher.go:316 +0x5c7
created by github.com/ethereum/go-ethereum/eth/fetcher.(*Fetcher).Start
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/fetcher/fetcher.go:178 +0x33

goroutine 109 [IO wait]:
internal/poll.runtime_pollWait(0x3714cd8, 0x72, 0x138f170)
        C:/go/src/runtime/netpoll.go:173 +0x4b
internal/poll.(*pollDesc).wait(0x13fa887c, 0x72, 0xbd4200, 0x0, 0x0)
        C:/go/src/internal/poll/fd_poll_runtime.go:85 +0x7d
internal/poll.(*ioSrv).ExecIO(0x19caad8, 0x13fa8794, 0x50247340, 0x2, 0x1, 0x0)
        C:/go/src/internal/poll/fd_windows.go:222 +0xf1
internal/poll.(*FD).acceptOne(0x13fa8780, 0x1214, 0x2fab07e0, 0x2, 0x2, 0x13fa8794, 0x30, 0x64b8490, 0x40ba2c, 0x8)
        C:/go/src/internal/poll/fd_windows.go:796 +0x82
internal/poll.(*FD).Accept(0x13fa8780, 0x29749708, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        C:/go/src/internal/poll/fd_windows.go:830 +0x119
net.(*netFD).accept(0x13fa8780, 0x2b4235a0, 0x30886240, 0x309ec928)
        C:/go/src/net/fd_windows.go:193 +0x60
net.(*TCPListener).accept(0x26b89ed0, 0x30886280, 0x602e9800, 0x43306d)
        C:/go/src/net/tcpsock_posix.go:139 +0x23
net.(*TCPListener).Accept(0x26b89ed0, 0x26b66f80, 0xc, 0x26bc0620, 0x7299ea)
        C:/go/src/net/tcpsock.go:260 +0x3c
net/http.(*Server).Serve(0x26a96780, 0x1392f50, 0x26b89ed0, 0x0, 0x0)
        C:/go/src/net/http/server.go:2826 +0x1c2
created by github.com/ethereum/go-ethereum/rpc.StartHTTPEndpoint
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/endpoints.go:50 +0x282

goroutine 261 [select]:
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fetchParts(0x13c8c000, 0x138c7d0, 0x13cfebe0, 0x13d20200, 0x266f6ea8, 0x25470140, 0x13c43f5c, 0x13c43f84, 0x13c43f7c, 0x13c43f74, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1140 +0x17f
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fetchBodies(0x13c8c000, 0x2bc576, 0x0, 0x1398360, 0x2cbc2fcc)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1069 +0x2dc
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func4(0x4, 0x11bbc3c)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:472 +0x34
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1(0x13c8c000, 0x28327480, 0x31a3e090)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:491 +0x46
created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:491 +0x8c

goroutine 262 [select]:
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fetchParts(0x13c8c000, 0x138c7d0, 0x13cfebe8, 0x13d20980, 0x266f6eb0, 0x25470180, 0x5bd91f5c, 0x5bd91f84, 0x5bd91f7c, 0x5bd91f74, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1140 +0x17f
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fetchReceipts(0x13c8c000, 0x2bc576, 0x0, 0xf2e12957, 0x29b5efcc)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1093 +0x2dc
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func5(0x4, 0x11bbc3c)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:473 +0x34
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1(0x13c8c000, 0x28327480, 0x31a3e0a0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:491 +0x46
created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:491 +0x8c

goroutine 5957 [select]:
github.com/ethereum/go-ethereum/p2p.(*Peer).pingLoop(0x4d05a7c0)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:252 +0xfe
created by github.com/ethereum/go-ethereum/p2p.(*Peer).run
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:206 +0xe4

goroutine 110 [chan receive, 11 minutes]:
github.com/ethereum/go-ethereum/cmd/utils.StartNode.func1(0x13d56140)
        C:/gopath/src/github.com/ethereum/go-ethereum/cmd/utils/cmd.go:74 +0xd3
created by github.com/ethereum/go-ethereum/cmd/utils.StartNode
        C:/gopath/src/github.com/ethereum/go-ethereum/cmd/utils/cmd.go:70 +0x9a

goroutine 111 [chan receive, 11 minutes]:
main.startNode.func1(0x13d56140, 0x25bfba00)
        C:/gopath/src/github.com/ethereum/go-ethereum/cmd/geth/main.go:311 +0x27a
created by main.startNode
        C:/gopath/src/github.com/ethereum/go-ethereum/cmd/geth/main.go:296 +0x21c

goroutine 112 [select, 11 minutes]:
net.(*pipe).read(0x26d3d590, 0x261aa600, 0x200, 0x200, 0x0, 0x43e569, 0x200)
        C:/go/src/net/pipe.go:164 +0x169
net.(*pipe).Read(0x26d3d590, 0x261aa600, 0x200, 0x200, 0x0, 0x200, 0x40ae5f)
        C:/go/src/net/pipe.go:147 +0x3d
encoding/json.(*Decoder).refill(0x25d622c0, 0xffffffe1, 0x1)
        C:/go/src/encoding/json/stream.go:159 +0xe9
encoding/json.(*Decoder).readValue(0x25d622c0, 0x0, 0x0, 0x13c30000)
        C:/go/src/encoding/json/stream.go:134 +0x1d4
encoding/json.(*Decoder).Decode(0x25d622c0, 0x105b1a0, 0x26bacdf0, 0x1, 0x26bacdf0)
        C:/go/src/encoding/json/stream.go:63 +0x64
encoding/json.(*Decoder).Decode-fm(0x105b1a0, 0x26bacdf0, 0x26a31930, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/json.go:119 +0x30
github.com/ethereum/go-ethereum/rpc.(*jsonCodec).ReadRequestHeaders(0x26a31920, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/json.go:144 +0xab
github.com/ethereum/go-ethereum/rpc.(*Server).readRequest(0x26b20d00, 0x13983a0, 0x26a31920, 0x19e8f01, 0x10c4b20, 0x26b9fec0, 0x13941c0, 0x26b9fee0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/server.go:375 +0x57
github.com/ethereum/go-ethereum/rpc.(*Server).serveRequest(0x26b20d00, 0x1394180, 0x13c2616c, 0x13983a0, 0x26a31920, 0x0, 0x3, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/server.go:160 +0x267
github.com/ethereum/go-ethereum/rpc.(*Server).ServeCodec(0x26b20d00, 0x13983a0, 0x26a31920, 0x3)
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/server.go:216 +0x77
created by github.com/ethereum/go-ethereum/rpc.DialInProc.func1
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/inproc.go:29 +0x9c

goroutine 113 [select, 11 minutes]:
github.com/ethereum/go-ethereum/rpc.(*Client).dispatch(0x25bfbac0, 0x1397de0, 0x26d3d5e0)
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/client.go:507 +0x257
created by github.com/ethereum/go-ethereum/rpc.newClient
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/client.go:206 +0x239

goroutine 146 [select, 11 minutes]:
net.(*pipe).read(0x26d3d5e0, 0x261aa400, 0x200, 0x200, 0x9a, 0x43e569, 0x200)
        C:/go/src/net/pipe.go:164 +0x169
net.(*pipe).Read(0x26d3d5e0, 0x261aa400, 0x200, 0x200, 0x0, 0x200, 0x1f)
        C:/go/src/net/pipe.go:147 +0x3d
encoding/json.(*Decoder).refill(0x25d62370, 0xfd6e60, 0x1380e87)
        C:/go/src/encoding/json/stream.go:159 +0xe9
encoding/json.(*Decoder).readValue(0x25d62370, 0x0, 0x0, 0x7ff)
        C:/go/src/encoding/json/stream.go:134 +0x1d4
encoding/json.(*Decoder).Decode(0x25d62370, 0x105b1a0, 0x26bacdc0, 0x0, 0x0)
        C:/go/src/encoding/json/stream.go:63 +0x64
github.com/ethereum/go-ethereum/rpc.(*Client).read.func1(0x1063a20, 0x26bacdd0, 0x26d3d5e0, 0x3649718, 0x26d3d5e0)
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/client.go:648 +0x62
github.com/ethereum/go-ethereum/rpc.(*Client).read(0x25bfbac0, 0x1397de0, 0x26d3d5e0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/client.go:661 +0xf5
created by github.com/ethereum/go-ethereum/rpc.(*Client).dispatch
        C:/gopath/src/github.com/ethereum/go-ethereum/rpc/client.go:482 +0x5a

goroutine 164 [select, 11 minutes]:
github.com/ethereum/go-ethereum/eth.(*peer).broadcast(0x297c4000)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/peer.go:116 +0x133
created by github.com/ethereum/go-ethereum/eth.(*peerSet).Register
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/peer.go:428 +0xfb

goroutine 6211 [chan receive]:
github.com/ethereum/go-ethereum/p2p/discover.(*udp).findnode(0x13d21340, 0x60917a24, 0xdf32bd02, 0x690dc1de, 0x6b1fb7b9, 0x1d09873a, 0x8faabe60, 0xd01511f0, 0x2f462cf9, 0x2b26e940, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/udp.go:366 +0x2e1
github.com/ethereum/go-ethereum/p2p/discover.(*Table).findnode(0x25672280, 0x58c031f0, 0xd1b97064, 0xf882110b, 0xace88d77, 0x3dcd27fe, 0xb6dcc490, 0xfe68371d, 0xcdf7e284, 0x44967434, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:311 +0x132
created by github.com/ethereum/go-ethereum/p2p/discover.(*Table).lookup
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:286 +0x40c

goroutine 5948 [select]:
github.com/ethereum/go-ethereum/eth.(*peer).broadcast(0x49d93180)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/peer.go:116 +0x133
created by github.com/ethereum/go-ethereum/eth.(*peerSet).Register
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/peer.go:428 +0xfb

goroutine 187 [chan receive, 11 minutes]:
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync(0x13c8c000, 0x26df4860, 0x5, 0x8, 0x5, 0x26df4860)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:502 +0xe7
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer(0x13c8c000, 0x13fb6000, 0xe383002, 0x842e0fe, 0x14ec1304, 0xc4495bcf, 0x81cc8326, 0x2ae72343, 0xd5fc0e83, 0xbd6ee23f, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:481 +0x602
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).synchronise(0x13c8c000, 0x296960a0, 0x10, 0xe383002, 0x842e0fe, 0x14ec1304, 0xc4495bcf, 0x81cc8326, 0x2ae72343, 0xd5fc0e83, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:406 +0x311
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).Synchronise(0x13c8c000, 0x296960a0, 0x10, 0xe383002, 0x842e0fe, 0x14ec1304, 0xc4495bcf, 0x81cc8326, 0x2ae72343, 0xd5fc0e83, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:324 +0x6b
github.com/ethereum/go-ethereum/eth.(*ProtocolManager).synchronise(0x13d12000, 0x297c4000)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/sync.go:200 +0x1a0
created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).syncer
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/sync.go:155 +0x166

goroutine 6214 [runnable]:
github.com/ethereum/go-ethereum/crypto/secp256k1._Cfunc_secp256k1_ext_ecdsa_recover(0x37733a8, 0x4dbe91d0, 0x580511a0, 0x54593320, 0x1)
        _cgo_gotypes.go:240 +0x3f
github.com/ethereum/go-ethereum/crypto/secp256k1.RecoverPubkey.func1(0x37733a8, 0x4dbe91d0, 0x580511a0, 0x54593320, 0x41)
        C:/gopath/src/github.com/ethereum/go-ethereum/crypto/secp256k1/secp256.go:106 +0x67
github.com/ethereum/go-ethereum/crypto/secp256k1.RecoverPubkey(0x54593320, 0x20, 0x20, 0x580511a0, 0x41, 0x41, 0x20, 0x61, 0x30f53300, 0xf, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/crypto/secp256k1/secp256.go:106 +0xb3
github.com/ethereum/go-ethereum/crypto.Ecrecover(0x54593320, 0x20, 0x20, 0x580511a0, 0x41, 0x41, 0x95aa6785, 0x6b81595d, 0xe9faa9fd, 0x54593260, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/crypto/signature_cgo.go:32 +0x49
github.com/ethereum/go-ethereum/consensus/clique.ecrecover(0x4ae68b40, 0x25676a20, 0x0, 0x0, 0x0, 0x0, 0x0, 0xf91eba5c, 0xd748a6a6)
        C:/gopath/src/github.com/ethereum/go-ethereum/consensus/clique/clique.go:188 +0x1e1
github.com/ethereum/go-ethereum/consensus/clique.(*Clique).verifySeal(0x25645450, 0x1396620, 0x25ad0060, 0x4ae68b40, 0x55458f00, 0xbf, 0xc0, 0xb2a88f9e, 0x36c2b6a8)
        C:/gopath/src/github.com/ethereum/go-ethereum/consensus/clique/clique.go:487 +0x100
github.com/ethereum/go-ethereum/consensus/clique.(*Clique).verifyCascadingFields(0x25645450, 0x1396620, 0x25ad0060, 0x4ae68b40, 0x55458f00, 0xbf, 0xc0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/consensus/clique/clique.go:374 +0x45e
github.com/ethereum/go-ethereum/consensus/clique.(*Clique).verifyHeader(0x25645450, 0x1396620, 0x25ad0060, 0x4ae68b40, 0x55458f00, 0xbf, 0xc0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/consensus/clique/clique.go:331 +0x361
github.com/ethereum/go-ethereum/consensus/clique.(*Clique).VerifyHeaders.func1(0x55458f00, 0xc0, 0xc0, 0x25645450, 0x1396620, 0x25ad0060, 0x26c70000, 0x2b9a2080)
        C:/gopath/src/github.com/ethereum/go-ethereum/consensus/clique/clique.go:259 +0x91
created by github.com/ethereum/go-ethereum/consensus/clique.(*Clique).VerifyHeaders
        C:/gopath/src/github.com/ethereum/go-ethereum/consensus/clique/clique.go:257 +0xa7

goroutine 210 [IO wait]:
internal/poll.runtime_pollWait(0x3714858, 0x72, 0x138f170)
        C:/go/src/runtime/netpoll.go:173 +0x4b
internal/poll.(*pollDesc).wait(0x13d423fc, 0x72, 0xbd4200, 0x0, 0x0)
        C:/go/src/internal/poll/fd_poll_runtime.go:85 +0x7d
internal/poll.(*ioSrv).ExecIO(0x19caad8, 0x13d42314, 0x11bb6a4, 0x1, 0x600, 0x4cd0a000)
        C:/go/src/internal/poll/fd_windows.go:222 +0xf1
internal/poll.(*FD).Read(0x13d42300, 0x63414d80, 0x20, 0x20, 0x0, 0x0, 0x0)
        C:/go/src/internal/poll/fd_windows.go:487 +0x1ec
net.(*netFD).Read(0x13d42300, 0x63414d80, 0x20, 0x20, 0xffffffe1, 0x0, 0x40b3fa)
        C:/go/src/net/fd_windows.go:152 +0x3f
net.(*conn).Read(0x287cf010, 0x63414d80, 0x20, 0x20, 0x0, 0x0, 0x0)
        C:/go/src/net/net.go:177 +0x56
io.ReadAtLeast(0x3718fa0, 0x287cf010, 0x63414d80, 0x20, 0x20, 0x20, 0x110cea0, 0x0, 0x3718fa0)
        C:/go/src/io/io.go:310 +0x78
io.ReadFull(0x3718fa0, 0x287cf010, 0x63414d80, 0x20, 0x20, 0x20, 0x19d04e0, 0x4903f6)
        C:/go/src/io/io.go:329 +0x45
github.com/ethereum/go-ethereum/p2p.(*rlpxFrameRW).ReadMsg(0x13c4b700, 0x77b38008, 0xbf10298e, 0x20db4c85, 0xab, 0x19d04e0, 0x0, 0x0, 0xbf10298e, 0x20db4c85, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/rlpx.go:648 +0xbd
github.com/ethereum/go-ethereum/p2p.(*rlpx).ReadMsg(0x287e9080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/rlpx.go:95 +0x105
github.com/ethereum/go-ethereum/p2p.(*Peer).readLoop(0x13fb4300, 0x13e5b540)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:268 +0x7b
created by github.com/ethereum/go-ethereum/p2p.(*Peer).run
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:205 +0xc3

goroutine 239 [select]:
github.com/ethereum/go-ethereum/eth/downloader.(*stateSync).loop(0x28327680, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/statesync.go:296 +0x1f5
github.com/ethereum/go-ethereum/eth/downloader.(*stateSync).run(0x28327680)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/statesync.go:255 +0x21
created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).runStateSync
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/statesync.go:106 +0xcd

goroutine 211 [select]:
github.com/ethereum/go-ethereum/p2p.(*Peer).pingLoop(0x13fb4300)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:252 +0xfe
created by github.com/ethereum/go-ethereum/p2p.(*Peer).run
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:206 +0xe4

goroutine 263 [runnable]:
github.com/ethereum/go-ethereum/core.(*HeaderChain).ValidateHeaderChain(0x25ad0060, 0x55458f00, 0xc0, 0xc0, 0x64, 0x0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/core/headerchain.go:246 +0x82f
github.com/ethereum/go-ethereum/core.(*BlockChain).InsertHeaderChain(0x24d68000, 0x55458f00, 0xc0, 0xc0, 0x64, 0x0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/core/blockchain.go:1588 +0x70
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processHeaders(0x13c8c000, 0x2c72b6, 0x0, 0x3aabf1, 0x0, 0x2d3356a0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1401 +0x5e6
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func6(0x4, 0x11bbc3c)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:474 +0x49
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1(0x13c8c000, 0x28327480, 0x26df4840)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:491 +0x46
created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:491 +0x8c

goroutine 212 [select]:
github.com/ethereum/go-ethereum/p2p.(*protoRW).ReadMsg(0x26a20b40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:415 +0xa4
github.com/ethereum/go-ethereum/eth.(*ProtocolManager).handleMsg(0x13d12000, 0x297c4000, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/handler.go:332 +0x6d
github.com/ethereum/go-ethereum/eth.(*ProtocolManager).handle(0x13d12000, 0x297c4000, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/handler.go:321 +0x72d
github.com/ethereum/go-ethereum/eth.NewProtocolManager.func1(0x13fb4300, 0x13900b0, 0x26a20b40, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/handler.go:148 +0x142
github.com/ethereum/go-ethereum/p2p.(*Peer).startProtocols.func1(0x26a20b40, 0x13fb4300, 0x13900b0, 0x26a20b40)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:361 +0x3b
created by github.com/ethereum/go-ethereum/p2p.(*Peer).startProtocols
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:360 +0x1cc

goroutine 260 [select]:
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fetchParts(0x13c8c000, 0x138c7d0, 0x13cfebd8, 0x13d20180, 0x27026740, 0x31060f80, 0x26e09ca4, 0x26e09cc4, 0x26e09cbc, 0x11baa84, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1140 +0x17f
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fillHeaderSkeleton(0x13c8c000, 0x2c2576, 0x0, 0x2a0f8480, 0x80, 0x8d, 0x3298c000, 0x6000, 0x6000, 0x6000, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1043 +0x304
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fetchHeaders(0x13c8c000, 0x13fb6000, 0x2c2576, 0x0, 0x3aabf1, 0x0, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:934 +0xfa3
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func3(0x4, 0x11bbc3c)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:471 +0x49
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1(0x13c8c000, 0x28327480, 0x26df4800)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:491 +0x46
created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:491 +0x8c

goroutine 265 [chan receive, 11 minutes]:
github.com/ethereum/go-ethereum/eth/downloader.(*stateSync).Wait(...)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/statesync.go:261
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processFastSyncContent.func1(0x267bc248, 0x13c8c000)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1513 +0x32
created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processFastSyncContent
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1512 +0xe3

goroutine 6156 [select]:
github.com/ethereum/go-ethereum/p2p/discover.(*Table).lookup(0x25672280, 0xd1b97064, 0xf882110b, 0xace88d77, 0x3dcd27fe, 0xb6dcc490, 0xfe68371d, 0xcdf7e284, 0x44967434, 0x6a701088, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:293 +0x4c2
github.com/ethereum/go-ethereum/p2p/discover.(*Table).LookupRandom(0x25672280, 0xbf102985, 0xc8953e71, 0xa2)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:244 +0x65
github.com/ethereum/go-ethereum/p2p.(*discoverTask).Do(0x2a9ae8d0, 0x13d8e6e0)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/dial.go:371 +0xcf
github.com/ethereum/go-ethereum/p2p.(*Server).run.func2.1(0x138ccb0, 0x2a9ae8d0, 0x13d8e6e0, 0x2567b680)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:638 +0x2d
created by github.com/ethereum/go-ethereum/p2p.(*Server).run.func2
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:638 +0x159

goroutine 6212 [chan receive]:
github.com/ethereum/go-ethereum/p2p/discover.(*udp).findnode(0x13d21340, 0x9ffd7924, 0x14146c14, 0x790a0b39, 0x5dc754, 0xeed717b0, 0x1f5b8571, 0x3454a7c6, 0x6591851a, 0x2b26ec60, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/udp.go:366 +0x2e1
github.com/ethereum/go-ethereum/p2p/discover.(*Table).findnode(0x25672280, 0x58c03260, 0xd1b97064, 0xf882110b, 0xace88d77, 0x3dcd27fe, 0xb6dcc490, 0xfe68371d, 0xcdf7e284, 0x44967434, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:311 +0x132
created by github.com/ethereum/go-ethereum/p2p/discover.(*Table).lookup
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:286 +0x40c

goroutine 6190 [IO wait]:
internal/poll.runtime_pollWait(0x3714b58, 0x72, 0x138f170)
        C:/go/src/runtime/netpoll.go:173 +0x4b
internal/poll.(*pollDesc).wait(0x2a3ac27c, 0x72, 0xbd4200, 0x0, 0x0)
        C:/go/src/internal/poll/fd_poll_runtime.go:85 +0x7d
internal/poll.(*ioSrv).ExecIO(0x19caad8, 0x2a3ac194, 0x11bb6a4, 0x2a3ac180, 0x57692200, 0x2ce9bd5c)
        C:/go/src/internal/poll/fd_windows.go:222 +0xf1
internal/poll.(*FD).Read(0x2a3ac180, 0x50048000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        C:/go/src/internal/poll/fd_windows.go:487 +0x1ec
net.(*netFD).Read(0x2a3ac180, 0x50048000, 0x1000, 0x1000, 0xffffffff, 0x0, 0x72191a)
        C:/go/src/net/fd_windows.go:152 +0x3f
net.(*conn).Read(0x29749700, 0x50048000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        C:/go/src/net/net.go:177 +0x56
net/http.(*connReader).Read(0x2b94b600, 0x50048000, 0x1000, 0x1000, 0x5b3d17e1, 0xbf, 0x19d04e0)
        C:/go/src/net/http/server.go:786 +0x12c
bufio.(*Reader).fill(0x2b59f6e0)
        C:/go/src/bufio/bufio.go:100 +0xf7
bufio.(*Reader).Peek(0x2b59f6e0, 0x4, 0xbf1029a4, 0x5b3d17e1, 0xbf, 0x19d04e0, 0x0)
        C:/go/src/bufio/bufio.go:132 +0x33
net/http.(*conn).serve(0x30886240, 0x1394160, 0x2b94b5e0)
        C:/go/src/net/http/server.go:1872 +0x785
created by net/http.(*Server).Serve
        C:/go/src/net/http/server.go:2851 +0x26a

goroutine 6112 [sleep]:
time.Sleep(0x1dcd6500, 0x0)
        C:/go/src/runtime/time.go:105 +0x14c
github.com/ethereum/go-ethereum/p2p/discover.(*udp).findnode(0x13d21340, 0x5ba67e24, 0x53bcde32, 0xc3d25d24, 0xc1f9e317, 0xb8471dd3, 0xd079c905, 0x3a33ff4d, 0x4031fe11, 0x2ac84640, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/udp.go:342 +0x3bd
github.com/ethereum/go-ethereum/p2p/discover.(*Table).findnode(0x25672280, 0x559d53b0, 0xd1b97064, 0xf882110b, 0xace88d77, 0x3dcd27fe, 0xb6dcc490, 0xfe68371d, 0xcdf7e284, 0x44967434, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:311 +0x132
created by github.com/ethereum/go-ethereum/p2p/discover.(*Table).lookup
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:286 +0x40c

goroutine 5956 [runnable]:
internal/poll.runtime_pollWait(0x3714ad8, 0x72, 0x138f170)
        C:/go/src/runtime/netpoll.go:173 +0x4b
internal/poll.(*pollDesc).wait(0x29d3377c, 0x72, 0xbd4200, 0x0, 0x0)
        C:/go/src/internal/poll/fd_poll_runtime.go:85 +0x7d
internal/poll.(*ioSrv).ExecIO(0x19caad8, 0x29d33694, 0x11bb6a4, 0x7080, 0x0, 0x0)
        C:/go/src/internal/poll/fd_windows.go:222 +0xf1
internal/poll.(*FD).Read(0x29d33680, 0x6415d941, 0x2290f, 0x2290f, 0x0, 0x0, 0x0)
        C:/go/src/internal/poll/fd_windows.go:487 +0x1ec
net.(*netFD).Read(0x29d33680, 0x6415d941, 0x2290f, 0x2290f, 0x7080, 0x0, 0x0)
        C:/go/src/net/fd_windows.go:152 +0x3f
net.(*conn).Read(0x28131998, 0x6415d941, 0x2290f, 0x2290f, 0x0, 0x0, 0x0)
        C:/go/src/net/net.go:177 +0x56
io.ReadAtLeast(0x3718fa0, 0x28131998, 0x6414c000, 0x34250, 0x34250, 0x34250, 0x110cea0, 0x0, 0x3718fa0)
        C:/go/src/io/io.go:310 +0x78
io.ReadFull(0x3718fa0, 0x28131998, 0x6414c000, 0x34250, 0x34250, 0x34250, 0x10, 0x634495c0)
        C:/go/src/io/io.go:329 +0x45
github.com/ethereum/go-ethereum/p2p.(*rlpxFrameRW).ReadMsg(0x516d5ec0, 0x56beab20, 0xbf10298e, 0xffe6779d, 0xaa, 0x19d04e0, 0x0, 0x0, 0xbf10298e, 0xffe6779d, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/rlpx.go:666 +0x263
github.com/ethereum/go-ethereum/p2p.(*rlpx).ReadMsg(0x17df8860, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/rlpx.go:95 +0x105
github.com/ethereum/go-ethereum/p2p.(*Peer).readLoop(0x4d05a7c0, 0x4d05aa80)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:268 +0x7b
created by github.com/ethereum/go-ethereum/p2p.(*Peer).run
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:205 +0xc3

goroutine 5958 [select]:
github.com/ethereum/go-ethereum/p2p.(*protoRW).ReadMsg(0x5a32dc70, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:415 +0xa4
github.com/ethereum/go-ethereum/eth.(*ProtocolManager).handleMsg(0x13d12000, 0x49d93180, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/handler.go:332 +0x6d
github.com/ethereum/go-ethereum/eth.(*ProtocolManager).handle(0x13d12000, 0x49d93180, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/handler.go:321 +0x72d
github.com/ethereum/go-ethereum/eth.NewProtocolManager.func1(0x4d05a7c0, 0x13900b0, 0x5a32dc70, 0x0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/eth/handler.go:148 +0x142
github.com/ethereum/go-ethereum/p2p.(*Peer).startProtocols.func1(0x5a32dc70, 0x4d05a7c0, 0x13900b0, 0x5a32dc70)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:361 +0x3b
created by github.com/ethereum/go-ethereum/p2p.(*Peer).startProtocols
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/peer.go:360 +0x1cc

goroutine 6183 [IO wait]:
internal/poll.runtime_pollWait(0x3714ed8, 0x72, 0x138f170)
        C:/go/src/runtime/netpoll.go:173 +0x4b
internal/poll.(*pollDesc).wait(0x64c02e7c, 0x72, 0xbd4200, 0x0, 0x0)
        C:/go/src/internal/poll/fd_poll_runtime.go:85 +0x7d
internal/poll.(*ioSrv).ExecIO(0x19caad8, 0x64c02d94, 0x11bb6a4, 0x11bbbc4, 0x5bd8da78, 0x454e30)
        C:/go/src/internal/poll/fd_windows.go:222 +0xf1
internal/poll.(*FD).Read(0x64c02d80, 0x2fab0620, 0xd2, 0xd2, 0x0, 0x0, 0x0)
        C:/go/src/internal/poll/fd_windows.go:487 +0x1ec
net.(*netFD).Read(0x64c02d80, 0x2fab0620, 0xd2, 0xd2, 0xffffffe1, 0x1ffffff, 0x40b3fa)
        C:/go/src/net/fd_windows.go:152 +0x3f
net.(*conn).Read(0x29749398, 0x2fab0620, 0xd2, 0xd2, 0x0, 0x0, 0x0)
        C:/go/src/net/net.go:177 +0x56
io.ReadAtLeast(0x3718fa0, 0x29749398, 0x2fab0620, 0xd2, 0xd2, 0xd2, 0x101dfa0, 0x1, 0x2fab0620)
        C:/go/src/io/io.go:310 +0x78
io.ReadFull(0x3718fa0, 0x29749398, 0x2fab0620, 0xd2, 0xd2, 0xd2, 0x64c02d8c, 0x40ae5f)
        C:/go/src/io/io.go:329 +0x45
github.com/ethereum/go-ethereum/p2p.readHandshakeMsg(0x138cc30, 0x58c03810, 0xd2, 0x13cfb200, 0x3718fa0, 0x29749398, 0x0, 0x0, 0x0, 0x0, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/rlpx.go:481 +0x7b
github.com/ethereum/go-ethereum/p2p.initiatorEncHandshake(0x3718f70, 0x29749398, 0x13cfb200, 0x527bcae0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/rlpx.go:296 +0x224
github.com/ethereum/go-ethereum/p2p.(*rlpx).doEncHandshake(0x2b5fed20, 0x13cfb200, 0x527bcae0, 0x0, 0x0, 0xad)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/rlpx.go:186 +0x95
github.com/ethereum/go-ethereum/p2p.(*Server).setupConn(0x13d8e6e0, 0x66137c20, 0x1, 0x58fda5b0, 0x527bcad0, 0x0)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:909 +0xae
github.com/ethereum/go-ethereum/p2p.(*Server).SetupConn(0x13d8e6e0, 0x1397d20, 0x29749398, 0x1, 0x58fda5b0, 0x4, 0x1397d20)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:884 +0xe9
github.com/ethereum/go-ethereum/p2p.(*dialTask).dial(0x66137bc0, 0x13d8e6e0, 0x58fda5b0, 0x445820d, 0xa737fae4)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/dial.go:354 +0x10f
github.com/ethereum/go-ethereum/p2p.(*dialTask).Do(0x66137bc0, 0x13d8e6e0)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/dial.go:297 +0x50
github.com/ethereum/go-ethereum/p2p.(*Server).run.func2.1(0x138cc90, 0x66137bc0, 0x13d8e6e0, 0x2567b680)
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:638 +0x2d
created by github.com/ethereum/go-ethereum/p2p.(*Server).run.func2
        C:/gopath/src/github.com/ethereum/go-ethereum/p2p/server.go:638 +0x159
ligi commented 5 years ago

Do you have an RPC port exposed to the public?

mmcc1 commented 5 years ago

Its just the default settings. So, perhaps. Are you suggesting that this is a denial of service attack?

ligi commented 5 years ago

yes this was my thinking - a lot of memory problems resulting from an open RPC and a lot of requests. But if you run on default settings this should not be the case.

mmcc1 commented 5 years ago

I am running with this command and I have tried removing syncmode and cache. Same result each time. 5 minutes then a crash.

geth --rinkeby --rpc --rpcapi="personal,eth,network,web3,net" --ipcpath "~/AppData/Roaming/Ethereum/geth.ipc" --syncmode "fast" --cache 2048

As far as I can see, its a 32 bit exe and its size is increasing. It looks like a memory leak to me.

ligi commented 5 years ago

but wait - then you are exposing rpc with personal. Does this machine have a public IP?

mmcc1 commented 5 years ago

Yes, it is on the web. Any logs I could look at?

mmcc1 commented 5 years ago

How do I disable listening on public IP addresses? I'm happy enough with loopback. I can test that and see if it resolves the problem.

ligi commented 5 years ago

can you elaborate on your use-case? Shouldn't IPC be enough?

mmcc1 commented 5 years ago

I haven't been able to get IPC to work. I used RPC on the loopback instead. I'm behind a NAT and I haven't setup any external ports, however, I do see my external IP address listed as the program starts.

ligi commented 5 years ago

ok if it is not exposed it should not be the problem. Just making sure as it is often the problem. perhaps @fjl has some ideas - seeing a lot of p2p stuff in the stacktrace.

mmcc1 commented 5 years ago

As I am syncing to the rinkeby network, I have had an opportunity to restart Geth about hundred times now. An interesting pattern is that the closer Geth gets to completing its sync, the faster out of memory errors are produced.

karalabe commented 5 years ago

Hmm, a 32 bit Windows binary can only ever use 3GB of ram. Why are you using a 32 bit binary on a machine with 32GB of RAM? Please use the 64 bit ones.

mmcc1 commented 5 years ago

I will try. I think the only installer packages are 32 bit.

karalabe commented 5 years ago

Ah, that's an issue with the downloads page. We have so many binaries, that listing them from azure returns multiple pages and we only process the first page. Will need that fixed. Sorry about that.

Here's a latest Windows 64 bit installer https://gethstore.blob.core.windows.net/builds/geth-windows-amd64-1.8.22-7fa3509e.exe

mmcc1 commented 5 years ago

I've tried the 64 bit version and so far running without issue. I'll run it overnight and see how it goes.

ligi commented 5 years ago

Great! I think we can close this issue then - @karalabe opened a follow up issue here #19042 Feel free to reopen if you run into OOM with the 64bit version