search

ethereum / go-ethereum

Go implementation of the Ethereum protocol
https://geth.ethereum.org
GNU Lesser General Public License v3.0
47.21k stars 19.99k forks source link

What is the upper bound of "imported new state entries"? When will it end? #15616

Closed idotial closed 1 year ago

idotial commented 6 years ago

System information

Geth version: 1.7.2 OS & Version: OSX

Expected behaviour

geth --fast should finish soon.

Actual behaviour

it run for 3days and print “Imported new state entries count=384 elapsed=26.970ms processed=50023987 pending=33074 retry=0 duplicate=19087 unexpected=47765” constantly

Steps to reproduce the behaviour

run geth --fast in console

Backtrace

INFO [12-06|07:08:00] Imported new state entries count=1259 elapsed=12.971ms processed=50014526 pending=34891 retry=0 duplicate=19087 unexpected=47765 INFO [12-06|07:08:23] Imported new state entries count=774 elapsed=8.950ms processed=50015300 pending=34311 retry=0 duplicate=19087 unexpected=47765 INFO [12-06|07:08:31] Imported new state entries count=1125 elapsed=9.513ms processed=50016425 pending=33428 retry=0 duplicate=19087 unexpected=47765 INFO [12-06|07:08:39] Imported new state entries count=1061 elapsed=11.198ms processed=50017486 pending=32566 retry=0 duplicate=19087 unexpected=47765 INFO [12-06|07:08:49] Imported new state entries count=1314 elapsed=12.041ms processed=50018800 pending=31248 retry=0 duplicate=19087 unexpected=47765 INFO [12-06|07:09:10] Imported new state entries count=1028 elapsed=10.446ms processed=50019828 pending=30496 retry=0 duplicate=19087 unexpected=47765 INFO [12-06|07:09:25] Imported new state entries count=1241 elapsed=10.423ms processed=50021069 pending=30088 retry=0 duplicate=19087 unexpected=47765 INFO [12-06|07:09:37] Imported new state entries count=777 elapsed=6.224ms processed=50021846 pending=29851 retry=26 duplicate=19087 unexpected=47765

nklak commented 6 years ago

geth --syncmode "fast" --cache 12288

that is 12GB for geth cache, but it is never over 30% of memory use.

100Jason commented 6 years ago

Just wanted to post this followup from my message back in January. About 2 months after being in sync and running 24 hours a day, I noticed it'd get behind and have to start syncing again. After a few hours it'd catch up and be in sync again, but eventually it hit the point where it was losing ground and not getting in sync anymore. This was despite having a good internet connection and an SSD on a computer that didn't do anything else. I got tired of hosting that huge database when it couldn't even keep it in sync anymore, so I gave up and figured out how to get the light mode working so I could at least get into my wallet. Haven't tried fast mode since.

stelloxx commented 6 years ago

Is 18+ days to build a "--fast" node norm? Looking around various comments it seems like either people sync-up within a week or not at all. Running almost 2wks on 200Mbit+ bandwidth, 100% dedicated hardware PowerEdge T110, Quad Xeon E3-1245 V2 @ 3.40GHz, 16GB RAM, w/ 500GB raid1 SATA (not SSD). Perpetually ~100 blocks behind with 217,857,984 knownStates so far. Running geth 1.8.13-stable-225171a4

nklak commented 6 years ago

OK, with SSD machine geth synced in less than 24h. I can not tell the exact number of state entries as I can not find geth log?!?

But main thing here is that it seems that HDDs can not keep up, and to use geth you need to run geth on SSDs. When using fast sync blockchain database is 120GB and I would like to know how will it grow after fast sync is finished

stelloxx commented 6 years ago

That's what I suspected, thanks! I'll try pushing geth data to an SSD and see how quickly it wraps up.

100Jason commented 6 years ago

nklak: After I was in sync back on January 30 I did a database capture (February 13th). That capture is 120 GB (so it grew from about 62 GB to 120 GB in 2 weeks). But I was running 1.7.3 at the time and it didn't have any sort of pruning. That was introduced with Iceberg. From reading the blog, it seems that the database will still grow, but how fast will be dependent upon the memory cache. I'd love to hear what size your database is in a couple of weeks.

nklak commented 6 years ago

100Jason, as I see it now, it is growing about 200-300MB a day...I will post more details later.

digiuzman commented 6 years ago

7 hours and i am on 69245052 🍶

andrewshvv commented 6 years ago

Successfully synced, it took us 3-4 days, fast sync, Digital Ocean 8 GB Memory / 160 GB SSD Disk + 1000 GB HDD - Ubuntu 16.04.4 x64 , last log entry was:

Imported new state entries count=331 elapsed=4.832ms processed=211975531 pending=0 retry=0 duplicate=46561 unexpected=178269
stelloxx commented 6 years ago

Sync'ed in a few hours on SSD drive. Was over 230M known states at the time.

pavel-main commented 5 years ago

Since there is no public information about current count of state entries (and I have no idea how to get that data from synced node), I'll post my latest SSD fast sync - took 24 hours, latest known state entries count was close to 240 million, geth version is 1.8.16-stable

ghost commented 5 years ago

Centralization inside data centers is not good for the network. I fear Ethereum has had its day. We need next-gen ledgers (e.g. DAGs) like never before.

cryptify commented 5 years ago

hynese:

Implying any non-State actor can run a "data center" good enough to get close to a 51% attack on the Ethereum network, let alone pull it off successfully and without notice

Lols

rarigita commented 5 years ago

Hi, with an SSD drive mounted on /mnt/crypto/ and the following command:

geth --syncmode "fast" --cache 2048 --datadir "/mnt/crypto/ethereum"

I'm waiting more than one week to fast sync. Any idea why?. People is reporting 24h sync with SSD drives. Any way to improve my sync? High bandwidth and dedicated server with 4Gb Ram, dual core CPU.

I'm still at 224429484 states. and more than 240000000 are reported.

top:

PID     USER   PR  NI    VIRT    RES     SHR   S  %CPU %MEM     TIME+    COMMAND                                                           
31312 xxx       20   0 2866744 1,662g  62608 S  48,8    44,5         23:39.21 geth

I had to restart geth several times.

rraallvv commented 5 years ago

I was able to sync a Bitcoin node with the snapshot here much faster than allowing it to download the data by itself, but can't find something like that for Ethereum. Does anyone know where to find the fast sync snapshot for Ethereum?

rarigita commented 5 years ago

Still at 240000000 and growing...

INFO [11-28|10:44:19.906] Imported new state entries count=507 elapsed=5.315ms processed=240236665 pending=3470 retry=0 duplicate=4627 unexpected=28978

rarigita commented 5 years ago

Still syncing:

> eth.syncing { currentBlock: 6793621, highestBlock: 6793734, knownStates: 261049222, pulledStates: 261040316, startingBlock: 6788824 }

rraallvv commented 5 years ago

@rarigita Could you please post the final number of pulledStates?

rraallvv commented 5 years ago

Finally got synced at 265036678 know states and block number 6815039

rarigita commented 5 years ago

My SSD drive went Read only itself, so I had to unmount, fsck, remount and rerun geth. Now I'm at:

{ currentBlock: 6800559, highestBlock: 6817840, knownStates: 262895563, pulledStates: 262895563, startingBlock: 6798915 }

rraallvv commented 5 years ago

@rarigita Thanks.

grittibaenz commented 5 years ago

It took me around 3 days and 4 hours to synchronize a ETH Geth fast node:

Settings: --syncmode fast --gcmode full --cache 4096 Start: 2018-12-15 10:00 UTC End: 2018-12-18 14:08 UTC State entries: 260172278 Disk usage: 149G Hardware: Virtual machine, 4 cores, 8 GB memory, 300 GB SSD

downpouring commented 5 years ago

01-02 10:36:56 --> 01-05 19:20:32 UTC+8 tencent cloud hk: C2.LARGE8 - 4C8GB 600GB HDD final entry: 267880510 disk usage: 150G sync mode: fast cache: 2048

rraallvv commented 5 years ago

For some reason mine has 217G disk usage, should I be worried about that?

Mr-zhangQi commented 5 years ago

It took me around 5 days to synchronize a ETH Geth fast node:

State entries: 274079704 Disk usage: 156G

jdrowell commented 5 years ago

Got it fast synced in around 4-5 days

Last pulledStates was over 279164294, db size 161GB

Needed two forced restarts because geth got into hung states (no new states coming in) for long periods (>10min). This was using a SSD and a residential 100Mbps connection.

mtj151 commented 5 years ago

Fast sync cache 2048 - 34ish hours sync.

i7 6600k 16gb ram m.2 NVME 1TB Used 171.1GB - measured a number of hours after sync completed.

Final State entries number - 277295920

e-scavo commented 5 years ago

In our case, after a long time and many #||@#!"|@ geth is synced! It took almost a week. SSD 50 GB 8GB RAM i5 Used 165GB of disk space. Final state entries number - 291739903

It is anoying, but at least it is synced!!

deke997 commented 5 years ago

Count as of 3/20/2019 is 292,494,140.

Rate of state entry import for the last 20M entries on my machine was 187M entries per 24 hours

https://github.com/ethereum/go-ethereum/issues/16558#issuecomment-475073726

https://github.com/ethereum/go-ethereum/issues/16558#issuecomment-475104906

state_entry

andrewheadricke commented 5 years ago

It would be great if the state database could be "on demand" instead of downloading the entire thing. Because I probably only care about < 1% of the total state information. Perhaps a new "light" mode that in addition to tracking block headers it downloads and stores state nodes "on demand" and compares state diffs only against state nodes my client has bothered to download.

Not sure if possible. But would be nice :)

mstrokin commented 5 years ago

4/2/2019 303.6M 172G

TuanAnh207 commented 5 years ago

My VM has been syncing for days :( eth.syncing {   currentBlock: 7550331,   highestBlock: 7550402,   knownStates: 314681698,   pulledStates: 314681256,   startingBlock: 7545995 } , any update on the current state of a synced client ?

cobordism commented 5 years ago

I have a linode 4GB (2 CPU cores, 4GB ram - https://www.linode.com/pricing) with an extra attached "high speed storage" device for geth.

I started geth with geth --syncmode=fast and I have been running it uninterrupted for 29 days so far.

$ geth attach --exec="eth.syncing"
{
  currentBlock: 7553668,
  highestBlock: 7553745,
  knownStates: 241426253,
  pulledStates: 241425376,
  startingBlock: 0
}

I guess I still have a few weeks to go before my fast sync completes.

At this point I just want to keep it running to see how long it takes until I sync all 314681698+ states... but clearly linodes are useless for running geth :(

nklak commented 5 years ago

On node with 16GB RAM and SSD storage fast sync takes about 2 days to complete. (more if your internet speed is slow). For GETH imperative is SSD, plain HDDs and even SAS can not keep up, so SSDs are only option if you want to run ethereum node.

e-scavo commented 5 years ago

I have a linode 4GB (2 CPU cores, 4GB ram - https://www.linode.com/pricing) with an extra attached "high speed storage" device for geth.

I started geth with geth --syncmode=fast and I have been running it uninterrupted for 29 days so far.

$ geth attach --exec="eth.syncing"
{
  currentBlock: 7553668,
  highestBlock: 7553745,
  knownStates: 241426253,
  pulledStates: 241425376,
  startingBlock: 0
}

I guess I still have a few weeks to go before my fast sync completes.

At this point I just want to keep it running to see how long it takes until I sync all 314681698+ states... but clearly linodes are useless for running geth :(

web3.eth.syncing { currentBlock: 7557761, highestBlock: 7561051, knownStates: 291739903, pulledStates: 291739903, startingBlock: 7554538 } web3.eth.syncing { currentBlock: 7557762, highestBlock: 7561051, knownStates: 291739903, pulledStates: 291739903, startingBlock: 7554538 } web3.eth.syncing { currentBlock: 7557764, highestBlock: 7561051, knownStates: 291739903, pulledStates: 291739903, startingBlock: 7554538 } web3.eth.syncing { currentBlock: 7557767, highestBlock: 7561051, knownStates: 291739903, pulledStates: 291739903, startingBlock: 7554538 }

mstrokin commented 5 years ago

When syncing, make sure you have enough peers you're connected to, e.g. in @e-scavo message we see that knownStates didn't change at all (not sure how long he/she waited between commands). From my experience, bottleneck is not HDD/SDD, but the network itself - sometimes it syncs states really fast, and sometimes it struggles to find a peer with enough states and gets stuck for hours.

Check your disk usage using iostat & iotop to see if geth is busy doing anything (as sometimes sync slows down due to db compression, which is expected) -- if you don't see any reads/writes for geth, restart the geth.

grittibaenz commented 5 years ago

I know I am not constructive, but it's just nuts to see how inefficient the whole Ethereum blockchain / apps (Geth and Parity) are designed. I don't understand why anyone would invest into this project if you have something like Bitcoin (which runs on a Raspberry Pi with dusty old spinning disks).

At the time of writing the ETH synced with Parity consumes >2.3 Terrabyte of data. I don't have a fully synced Geth node to give the numbers here, but it's probably even more. Anyone knows?

¯\_(ツ)_/¯

nklak commented 5 years ago

Geth node needs some 24-48h to fast-sync with SSD and good network connection. Currently it takes some 200GB of space. There is no need for anyone to run full sync so you are just wasting 2+ TB of data.

When running geth make sure that you define larger buffer for database (cache option). This is my geth startup.

geth -rpc -rpcapi eth,personal,net,web3 -cache 16384

Best regards

grittibaenz commented 5 years ago

There are business cases where you need the full blockchain to analyze the history (AML...)

cobordism commented 5 years ago

grittibaenz please stop. It is getting tiresome how often people post this misinformation.

A full node contains all the data (~200GB).

See here for proof: https://medium.com/@marcandrdumas/are-ethereum-full-nodes-really-full-an-experiment-b77acd086ca7

@karalabe or @fjl If you know of an online resource that shows the current number of knownStates, then I suggest we post the link here and close the issue. (and also post at issue 14647) The only node-stats sites I can find list blocknumber and diskspace, but not number of state entries.

[or can we add the number of known state to ethstats.net?]

rexey3s commented 5 years ago

Update on 4/17/2019 The number of pulled states is around 305M just before it is synced Disk: 176 GB Running geth with larger cache (mine is 4096) would speed up the synchronization.

wh0am111 commented 5 years ago

Update on 4/19/2019 The number of pulled states is around 318532338 when it is synced Disk: 178G

aihua commented 5 years ago

1) pulledStates=306397423 / fast sync from 04-19 09:32, complete on 04-21 08:42:03 / storage: 180GB / memory 16GB / CPU: AMD Athlon(tm) X4 870K Quad Core Processor / WDC WDS100T2B0A SSD/Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz /geth --cache=4096 --syncmode "fast"

2) pulledStates=310416897 / fast sync from: 04-19 09:32, complete on: 04-22 19:55:27 / storage: 180GB / memory: 8G B/ cpu: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz / SSD / Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz / geth --cache=4096 --syncmode "fast"

Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Upgrading chain index type=bloombits percentage=99 Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Imported new block headers count=1 elapsed=7.829ms number=7617253 hash=5519b7…b14245 age=1m43s Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.213] Imported new block headers count=2 elapsed=14.470ms number=7617255 hash=1b2c47…cb5ae0 age=1m7s Apr 22 19:56:44 eth-001 geth[2012]: INFO [04-22|19:56:44.592] Imported new chain segment blocks=1 txs=205 mgas=7.991 elapsed=1.372s mgasps=5.822 number=7617252 hash=5efb08…180758 age=2m48s cache=87.82mB Apr 22 19:56:46 eth-001 geth[2012]: INFO [04-22|19:56:46.449] Finished upgrading chain index type=bloombits Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new chain segment blocks=3 txs=362 mgas=23.964 elapsed=3.453s mgasps=6.938 number=7617255 hash=1b2c47…cb5ae0 age=1m12s cache=91.15mB Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new block headers count=3 elapsed=4.827s number=7617258 hash=15ffda…93cff9 Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.078] Imported new chain segment blocks=3 txs=276 mgas=17.896 elapsed=2.016s mgasps=8.875 number=7617258 hash=15ffda…93cff9 cache=93.46mB Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.079] Fast sync complete, auto disabling Apr 22 19:56:53 eth-001 geth[2012]: INFO [04-22|19:56:53.150] Imported new chain segment blocks=1 txs=3 mgas=8.000 elapsed=1.762s mgasps=4.538 number=7617259 hash=6279e1…69d3ff cache=93.95mB

wh0am111 commented 5 years ago
  1. pulledStates=306397423 / fast sync from 04-19 09:32, complete on 04-21 08:42:03 / storage: 180GB / memory 16GB / CPU: AMD Athlon(tm) X4 870K Quad Core Processor / WDC WDS100T2B0A SSD/Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz /geth --cache=4096 --syncmode "fast"
  2. pulledStates=310416897 / fast sync from: 04-19 09:32, complete on: 04-22 19:55:27 / storage: 180GB / memory: 8G B/ cpu: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz / SSD / Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz / geth --cache=4096 --syncmode "fast"

Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Upgrading chain index type=bloombits percentage=99 Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Imported new block headers count=1 elapsed=7.829ms number=7617253 hash=5519b7…b14245 age=1m43s Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.213] Imported new block headers count=2 elapsed=14.470ms number=7617255 hash=1b2c47…cb5ae0 age=1m7s Apr 22 19:56:44 eth-001 geth[2012]: INFO [04-22|19:56:44.592] Imported new chain segment blocks=1 txs=205 mgas=7.991 elapsed=1.372s mgasps=5.822 number=7617252 hash=5efb08…180758 age=2m48s cache=87.82mB Apr 22 19:56:46 eth-001 geth[2012]: INFO [04-22|19:56:46.449] Finished upgrading chain index type=bloombits Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new chain segment blocks=3 txs=362 mgas=23.964 elapsed=3.453s mgasps=6.938 number=7617255 hash=1b2c47…cb5ae0 age=1m12s cache=91.15mB Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new block headers count=3 elapsed=4.827s number=7617258 hash=15ffda…93cff9 Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.078] Imported new chain segment blocks=3 txs=276 mgas=17.896 elapsed=2.016s mgasps=8.875 number=7617258 hash=15ffda…93cff9 cache=93.46mB Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.079] Fast sync complete, auto disabling Apr 22 19:56:53 eth-001 geth[2012]: INFO [04-22|19:56:53.150] Imported new chain segment blocks=1 txs=3 mgas=8.000 elapsed=1.762s mgasps=4.538 number=7617259 hash=6279e1…69d3ff cache=93.95mB

180GB？ I think is not enough

aihua commented 5 years ago
  1. pulledStates=306397423 / fast sync from 04-19 09:32, complete on 04-21 08:42:03 / storage: 180GB / memory 16GB / CPU: AMD Athlon(tm) X4 870K Quad Core Processor / WDC WDS100T2B0A SSD/Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz /geth --cache=4096 --syncmode "fast"
  2. pulledStates=310416897 / fast sync from: 04-19 09:32, complete on: 04-22 19:55:27 / storage: 180GB / memory: 8GB/ cpu: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz / SSD / Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz / geth --cache=4096 --syncmode "fast"

Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Upgrading chain index type=bloombits percentage=99 Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Imported new block headers count=1 elapsed=7.829ms number=7617253 hash=5519b7…b14245 age=1m43s Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.213] Imported new block headers count=2 elapsed=14.470ms number=7617255 hash=1b2c47…cb5ae0 age=1m7s Apr 22 19:56:44 eth-001 geth[2012]: INFO [04-22|19:56:44.592] Imported new chain segment blocks=1 txs=205 mgas=7.991 elapsed=1.372s mgasps=5.822 number=7617252 hash=5efb08…180758 age=2m48s cache=87.82mB Apr 22 19:56:46 eth-001 geth[2012]: INFO [04-22|19:56:46.449] Finished upgrading chain index type=bloombits Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new chain segment blocks=3 txs=362 mgas=23.964 elapsed=3.453s mgasps=6.938 number=7617255 hash=1b2c47…cb5ae0 age=1m12s cache=91.15mB Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new block headers count=3 elapsed=4.827s number=7617258 hash=15ffda…93cff9 Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.078] Imported new chain segment blocks=3 txs=276 mgas=17.896 elapsed=2.016s mgasps=8.875 number=7617258 hash=15ffda…93cff9 cache=93.46mB Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.079] Fast sync complete, auto disabling Apr 22 19:56:53 eth-001 geth[2012]: INFO [04-22|19:56:53.150] Imported new chain segment blocks=1 txs=3 mgas=8.000 elapsed=1.762s mgasps=4.538 number=7617259 hash=6279e1…69d3ff cache=93.95mB

180GB？ I think is not enough

~180GB is used when completing fast sync, ssd storage is 1TB.

wh0am111 commented 5 years ago

Ooh，nice

发送自 Windows 10 版邮件https://go.microsoft.com/fwlink/?LinkId=550986应用

发件人: Aihua notifications@github.com 发送时间: Tuesday, April 23, 2019 9:24:04 AM 收件人: ethereum/go-ethereum 抄送: wh0am111; Comment 主题: Re: [ethereum/go-ethereum] What is the upper bound of "imported new state entries"? When will it end? (#15616)

  1. pulledStates=306397423 / fast sync from 04-19 09:32, complete on 04-21 08:42:03 / storage: 180GB / memory 16GB / CPU: AMD Athlon(tm) X4 870K Quad Core Processor / WDC WDS100T2B0A SSD/Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz /geth --cache=4096 --syncmode "fast"
  2. pulledStates=310416897 / fast sync from: 04-19 09:32, complete on: 04-22 19:55:27 / storage: 180GB / memory: 8G B/ cpu: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz / SSD / Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz / geth --cache=4096 --syncmode "fast"

Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Upgrading chain index type=bloombits percentage=99 Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Imported new block headers count=1 elapsed=7.829ms number=7617253 hash=5519b7…b14245 age=1m43s Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.213] Imported new block headers count=2 elapsed=14.470ms number=7617255 hash=1b2c47…cb5ae0 age=1m7s Apr 22 19:56:44 eth-001 geth[2012]: INFO [04-22|19:56:44.592] Imported new chain segment blocks=1 txs=205 mgas=7.991 elapsed=1.372s mgasps=5.822 number=7617252 hash=5efb08…180758 age=2m48s cache=87.82mB Apr 22 19:56:46 eth-001 geth[2012]: INFO [04-22|19:56:46.449] Finished upgrading chain index type=bloombits Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new chain segment blocks=3 txs=362 mgas=23.964 elapsed=3.453s mgasps=6.938 number=7617255 hash=1b2c47…cb5ae0 age=1m12s cache=91.15mB Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new block headers count=3 elapsed=4.827s number=7617258 hash=15ffda…93cff9 Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.078] Imported new chain segment blocks=3 txs=276 mgas=17.896 elapsed=2.016s mgasps=8.875 number=7617258 hash=15ffda…93cff9 cache=93.46mB Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.079] Fast sync complete, auto disabling Apr 22 19:56:53 eth-001 geth[2012]: INFO [04-22|19:56:53.150] Imported new chain segment blocks=1 txs=3 mgas=8.000 elapsed=1.762s mgasps=4.538 number=7617259 hash=6279e1…69d3ff cache=93.95mB

180GB？ I think is not enough

~180GB is used when completing fast sync, ssd storage is 1TB.

― You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ethereum/go-ethereum/issues/15616#issuecomment-485607035, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKUHGT2RU4WWZ2ILP5X6AWLPRZQLJANCNFSM4EG7I5CQ.

hayorov commented 5 years ago

My statistics:

2019-05-06 01:00:32 avg: 1827 max: 1938 min: 1378 states/s  remain: 136604075 states     4 peers    eta@ 20:46:28.165828
2019-05-06 01:00:37 avg: 1864 max: 1938 min: 1378 states/s  remain: 136595500 states     3 peers    eta@ 20:21:14.951050
2019-05-06 01:00:42 avg: 1791 max: 1938 min: 1378 states/s  remain: 136583359 states     3 peers    eta@ 21:11:16.481006
2019-05-06 01:00:48 avg: 1742 max: 1938 min: 1378 states/s  remain: 136580287 states     3 peers    eta@ 21:46:35.797305
2019-05-06 01:00:53 avg: 1721 max: 1938 min: 1378 states/s  remain: 136575694 states     3 peers    eta@ 22:03:01.154434
2019-05-06 01:00:58 avg: 1682 max: 1938 min: 1378 states/s  remain: 136569043 states     4 peers    eta@ 22:33:15.402442
2019-05-06 01:01:03 avg: 1698 max: 1938 min: 1378 states/s  remain: 136564293 states     3 peers    eta@ 22:20:27.458747

PS. I wrote a tiny python script to overview the process. It's here https://github.com/hayorov/ethereum-sync-mertics

andrewheadricke commented 5 years ago

Roughly 368,609,457 State entries @ Block 8,432,213 Chaindata = 70GiB Ancient = 93GiB

Really wish there was a syncmode = selfish where I can run a light node but keep up to date state/event information for just accounts / contracts I interact with.

deke997 commented 4 years ago

10/11/2019 379,265,938 state entries 187.5 GB of disk space used for fast sync

momaee commented 4 years ago

12-25-2019 Geth version: 1.9.9 Highest block: 9160798 Known states: 475519190