Tips for running a BSC full node

unclezoro commented 2 years ago

Some of the enhancements below can address the existing challenges with running a BSC full node:

Binary

All the clients are suggested to upgrade to the latest release. The latest version is supposed to be more stable and get better performance.

Storage

According to the test, the performance of a fullnoded will degrade when the storage size exceeds 1.5T. We suggest the fullnode always keeps light storage by pruning the storage.

Following are the steps to do prune:

Stop the BSC node first.
Run nohup geth snapshot prune-state --datadir {the data dir of your bsc node} &. It will take 3-5 hours to finish.
Start the node once it is done.

The maintainers should always have a few backup nodes so that you can switch to the backup ones when one of them is pruning.

The hardware is also important, make sure the SSD meets: 2T GB of free disk space, solid-state drive(SSD), gp3, 8k IOPS, 250MB/S throughput, read latency <1ms.

Light Storage

When the node crashes or is force killed, the node will sync from a block that was a few minutes or a few hours ago. This is because the state in memory is not persisted into the database in real time, and the node needs to replay blocks from the last checkpoint. The replaying time dependents on the configuration TrieTimeout in the config.toml. We suggest you raise it if you can tolerate with long replaying time, so the node can keep light storage.

Performance Tuning

In the logs, mgasps means the block processing ability of the fullnode, make sure the value is above 50.

The node can enable the profile function by —pprof

Profile by curl -sK -v http://127.0.0.1:6060/debug/pprof/profile?seconds=60 > profile_60s.out, and the dev community can help to analyze the performance.

New Node

If you build a new BSC node, please fetch snapshot from: https://github.com/binance-chain/bsc-snapshots

interzone2 commented 2 years ago

Finally, my full node is in-sync. I tried a lot of cloud providers (AWS/DO) to set this up. I managed to get my node in-sync with a dedicated server from Hetzner. It took 6 hours with a snapshot. Ping me if you need the IP address to bootstrap your node, I need to whitelist your IP address though because it is only being used internally now.

Hi, I am on Hetzner as well but on a managed server which unfortunately has a lot of restrictions. Can you, please, post the command lines used to sync the snapshot after the download? Thanks!

Hey Marian, do not forget to download the latest geth version and snapshot. I used a Storage box to temporarily store the snapshot which I extracted to the root disk. Anyway, here you go: /bsc/geth_linux --cache 75000 --nat=extip:<external-ip> --port 30413 --datadir /bsc/node --ws --ws.origins '*' --ws.api eth,net,web3 --ws.addr "<external-ip>" --ws.port 8545 --http --http.api net,personal,eth,web3 --http.addr 0.0.0.0 --allow-insecure-unlock --txlookuplimit=0 --config /bsc/config.toml --snapshot=true --maxpeers 100 --rpc.allow-unprotected-txs --diffsync

I use tmux to run it in the background.

This is what I changed in my config.toml [Node.P2P] MaxPeers = 100 DiscoveryV5 = true

Hey, I'm following your very helpful instructions. I'm also on a Hetzner machine with storage box. Could you tell me how did you unpack the archive to the root directory? Did you do this from within an SSH or SFTP session or was it by mounting the disc? Any tips much appreciated! Thanks.

osatien commented 2 years ago

the log says looking for peer (it has 16) and eth.syncing is false ... the blocknumber is the last of snapshot, and i dont see anything more happening :( snapshot generated ... and after that i am here :/ any help please¿

du5 commented 2 years ago

All the clients are suggested to upgrade to the latest release. The latest version is supposed to be more stable and get better performance.

@guagualvcha Maybe you can use https://github.com/binance-chain/bsc/releases/latest

marianfurdui commented 2 years ago

@guagualvcha I deleted everything and started from scratch.

-dowloaded and extracted latest snapshot: v geth-20211208.tar.gz
-downloaded latest geth: v1.1.7
-running get using: ./bscmainnet/geth_linux --config ./bscmainnet/config.toml --datadir ./bscmainnet/ --diffsync --cache 80000 --rpc.allow-unprotected-txs --txlookuplimit 0

the logs (from bsc.log)are the same...

t=2021-12-11T11:53:18+0100 lvl=info msg="Looking for peers"       peercount=0 tried=0  static=39
t=2021-12-11T11:53:53+0100 lvl=info msg="Looking for peers"        peercount=0 tried=39 static=39
t=2021-12-11T11:54:04+0100 lvl=info msg="Looking for peers"        peercount=0 tried=42 static=39
t=2021-12-11T11:54:28+0100 lvl=info msg="Looking for peers"        peercount=0 tried=0  static=39
t=2021-12-11T11:55:03+0100 lvl=info msg="Looking for peers"        peercount=0 tried=39 static=39
t=2021-12-11T11:55:14+0100 lvl=info msg="Looking for peers"        peercount=0 tried=42 static=39
t=2021-12-11T11:55:38+0100 lvl=info msg="Looking for peers"        peercount=0 tried=0  static=39

how to handle this case? Thanks!

hauchu1196 commented 2 years ago

@mj-dcb I want run fullnode in hetzner. Are you recommend server? Do you have guide for setup it?

mj-dcb commented 2 years ago

@mj-dcb I want run fullnode in hetzner. Are you recommend server? Do you have guide for setup it?

I recommend you to use AX61-NVMe (2TB). You should follow the steps from https://docs.binance.org/smart-chain/developer/fullnode.html and be sure to use the latest binary, it has improvements in--diffsync's function. Let me know if you need guidance, I help crypto projects set up a BSC fullnode but it's a paid job due to limited time.

JohnsonCaii commented 2 years ago

no snapshot paired state

Same here, any suggestions?

interzone2 commented 2 years ago

@mj-dcb I want run fullnode in hetzner. Are you recommend server? Do you have guide for setup it?

I recommend you to use AX61-NVMe (2TB). You should follow the steps from https://docs.binance.org/smart-chain/developer/fullnode.html and be sure to use the latest binary, it has improvements in--diffsync's function. Let me know if you need guidance, I help crypto projects set up a BSC fullnode but it's a paid job due to limited time.

Do you think that migrating to an AX61-NVMe (2x2TB) machine from an AX51-NVMe (2x1TB) box that is running well, still importing new state entries, but remains perpetually around 100 blocks behind - would do it? The bump in CPU and RAM is enough to solve this lag and the prime factor on why it can't sync?

guiltylelouch commented 2 years ago

mgasps only 13，what happen and how to improve it.

profile here profile_60s.txt

psdlt commented 2 years ago

@guiltylelouch you also have snapshot generation running at the same time. mgasps will be low during that. wait until snapshot generation is done and mgasps will improve.

nilampatel-engineer commented 2 years ago

How do we know snapshot is generating

Get Outlook for Androidhttps://aka.ms/AAb9ysg

From: Vaidas @.> Sent: Tuesday, December 21, 2021 3:30:50 PM To: binance-chain/bsc @.> Cc: nilampatel-engineer @.>; Comment @.> Subject: Re: [binance-chain/bsc] Tips for running a BSC full node (Issue #502)

@guiltylelouchhttps://github.com/guiltylelouch you also have snapshot generation running at the same time. mgasps will be low during that. wait until snapshot generation is done and mgasps will improve.

— Reply to this email directly, view it on GitHubhttps://github.com/binance-chain/bsc/issues/502#issuecomment-998873447, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ATHSSIQCKSY7TGBZOYQVJXTUSCMSVANCNFSM5HEABJYQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you commented.Message ID: @.***>

guiltylelouch commented 2 years ago

Do i need to run snapshot prune-state？

psdlt commented 2 years ago

@nilampatel-engineer you see aborting/resuming state snapshot generation in geth logs. see above screenshot for reference.

psdlt commented 2 years ago

@guiltylelouch no, you only need to wait until you no longer see messages in logs about state snapshot generation.

KynnoEnterprise commented 2 years ago

Hello,

I am going to create my BSC full node for the first time. I rented a high performance server to not be limited, but I just read your comments and it does not reassure me .

Does anyone have a doc or tutorial that would prevent me from starting in the wrong direction?

I thank you in advance.

mj-dcb commented 2 years ago

Hello,

I am going to create my BSC full node for the first time. I rented a high performance server to not be limited, but I just read your comments and it does not reassure me .

Does anyone have a doc or tutorial that would prevent me from starting in the wrong direction?

I thank you in advance.

Please follow https://docs.binance.org/smart-chain/developer/fullnode.html. Do not sync from block 0 but use the snapshot instead. If the node has enough resources it will take up to a day to process the snapshot and all state entries.

edit: also use the latest binary v1.1.7, it looks like this version performs better than the previous versions.

KynnoEnterprise commented 2 years ago

Hello, I am going to create my BSC full node for the first time. I rented a high performance server to not be limited, but I just read your comments and it does not reassure me . Does anyone have a doc or tutorial that would prevent me from starting in the wrong direction? I thank you in advance.

Please follow https://docs.binance.org/smart-chain/developer/fullnode.html. Do not sync from block 0 but use the snapshot instead. If the node has enough resources it will take up to a day to process the snapshot and all state entries.

edit: also use the latest binary v1.1.7, it looks like this version performs better than the previous versions.

I think I succeeded yesterday, currently it synchronizes, now if I want to retrieve the link to connect to the node, is there a documentation source? I have tested via port 8545 but so far I can't get it to work. Maybe I have to wait until the synchronization is finished.

I will continue to search but if someone here has a solution, I accept it :)

mj-dcb commented 2 years ago

Hello, I am going to create my BSC full node for the first time. I rented a high performance server to not be limited, but I just read your comments and it does not reassure me . Does anyone have a doc or tutorial that would prevent me from starting in the wrong direction? I thank you in advance.

Please follow https://docs.binance.org/smart-chain/developer/fullnode.html. Do not sync from block 0 but use the snapshot instead. If the node has enough resources it will take up to a day to process the snapshot and all state entries. edit: also use the latest binary v1.1.7, it looks like this version performs better than the previous versions.

I think I succeeded yesterday, currently it synchronizes, now if I want to retrieve the link to connect to the node, is there a documentation source? I have tested via port 8545 but so far I can't get it to work. Maybe I have to wait until the synchronization is finished.

I will continue to search but if someone here has a solution, I accept it :)

I think you forgot a few params in your CLI command. Did you add --http or --ws with its required params? You need to define the public IP and port to allow incoming connections. Do not forget to add incoming rules in the firewall if you have any.

Tonyladz commented 2 years ago

Hi guys, i'm running AX101 Hetzner dedicated root server, my friend is getting over 50-100 mgasps and also has the same dedicated root server. But I'm literally getting 3 mgasps. I dont know what im doing wrong. Can someone assist? I'm trying to set up a full node. It's been syncing for 4 days and not completed yet. I'm roughly 12000 blocks behind.

Untitled

osatien commented 2 years ago

Try to start without snapshot (--snapshot=false???) .... I think that your node is trying to do the snapshot and it can't

El dom., 26 dic. 2021 10:11, Tonyladz @.***> escribió:

Hi guys, i'm running AX101 Hetzner dedicated root server, my friend is getting over 50-100 mgasps and he has the same server. But I'm literally getting 10 mgasps. I dont know what im doing wrong. Can someone assist? I'm trying to run a full node.

— Reply to this email directly, view it on GitHub https://github.com/binance-chain/bsc/issues/502#issuecomment-1001135487, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABV7I75XOAODE2VWVV5ROHTUS3L3NANCNFSM5HEABJYQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

Tonyladz commented 2 years ago

mgasps only 13，what happen and how to improve it.

profile here profile_60s.txt

Did you end up fixing it? My mgasps is literally around 3 at the moment and i'm running hetzner top the line dedicated root server.

mj-dcb commented 2 years ago

Can you share your CLI cmd?

Op 26 dec. 2021 om 11:37 heeft Tonyladz @.***> het volgende geschreven:

mgasps only 13，what happen and how to improve it.

profile here profile_60s.txt

Did you end up fixing it? My mgasps is literally around 3 at the moment and i'm running hetzner top the line dedicated root server.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.

Tonyladz commented 2 years ago

Try to start without snapshot (--snapshot=false???) .... I think that your node is trying to do the snapshot and it can't El dom., 26 dic. 2021 10:11, Tonyladz @.> escribió: … Hi guys, i'm running AX101 Hetzner dedicated root server, my friend is getting over 50-100 mgasps and he has the same server. But I'm literally getting 10 mgasps. I dont know what im doing wrong. Can someone assist? I'm trying to run a full node. — Reply to this email directly, view it on GitHub <#502 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABV7I75XOAODE2VWVV5ROHTUS3L3NANCNFSM5HEABJYQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you commented.Message ID: @.>

I'm running this in start.sh ./geth_linux --config ./config.toml --datadir ./mainnet --cache 18000 --rpc.allow-unprotected-txs --txlookuplimit 0 --http --maxpeers 100 --ws --syncmode=full --snapshot=false --diffsync

& config.toml

[Eth] NetworkId = 56 NoPruning = false NoPrefetch = false LightPeers = 100 UltraLightFraction = 75 TrieTimeout = 100000000000 EnablePreimageRecording = false EWASMInterpreter = "" EVMInterpreter = ""

[Eth.Miner] GasFloor = 30000000 GasCeil = 40000000 GasPrice = 1000000000 Recommit = 10000000000 Noverify = false

[Eth.TxPool] Locals = [] NoLocals = true Journal = "transactions.rlp" Rejournal = 3600000000000 PriceLimit = 1000000000 PriceBump = 10 AccountSlots = 512 GlobalSlots = 10000 AccountQueue = 256 GlobalQueue = 5000 Lifetime = 10800000000000

[Eth.GPO] Blocks = 20 Percentile = 60 OracleThreshold = 20

[Node] IPCPath = "geth.ipc" HTTPHost = "localhost" NoUSB = true InsecureUnlockAllowed = false HTTPPort = 8545 HTTPVirtualHosts = ["localhost"] HTTPModules = ["eth", "net", "web3", "txpool", "parlia"] WSPort = 8546 WSModules = ["net", "web3", "eth"]

[Node.P2P] MaxPeers = 30 NoDiscovery = false BootstrapNodes = ["enode://1cc4534b14cfe351ab740a1418ab944a234ca2f702915eadb7e558a02010cb7c5a8c295a3b56bcefa7701c07752acd5539cb13df2aab8ae2d98934d712611443@52.71.43.172:30311","enode://28b1d16562dac280dacaaf45d54516b85bc6c994252a9825c5cc4e080d3e53446d05f63ba495ea7d44d6c316b54cd92b245c5c328c37da24605c4a93a0d099c4@34.246.65.14:30311","enode://5a7b996048d1b0a07683a949662c87c09b55247ce774aeee10bb886892e586e3c604564393292e38ef43c023ee9981e1f8b335766ec4f0f256e57f8640b079d5@35.73.137.11:30311"] StaticNodes = ["enode://b208ecff0e78b0de84fbed72ce4a39590903ead22759dc77b560298550394dd723b232daa42c76596a6763447dc3a7951027cc5fb3f8085d42775023c9412c63@23.21.154.239:30311", "enode://bcccac7dfbd21bd1dbbb7bf64ef7af6986520091f00320905e5919d12167fc8c94698099946933747ad2ea24745d35ebc980f9ba1475a839c83574d5ccf318d9@34.196.94.250:30311", "enode://8661a9799cb9c202c87c895b06de689279fb3b9795b7a00a33be04d0651331a3f0603b1056fb2a8a9eeea76a4396f6fa8d29d03b4f3f29f34adf33fe0e96d905@44.197.67.153:30311", "enode://3fe3aca8482301a3b966e40e66b9a924ede7bb10b6f76e2c1c0aa421e293189e44d381c652d9f875ea7f865829e9a8e6a5d006e5a5e54176aa2069dd18b18ef6@23.22.69.150:30311", "enode://29af5f724433add3047a2f20b4c82ca587fe507c579531798a4bad25cbcbb0243d9da94d14f8388b80a7780568db39c1bee7a8862b8bebe7f8c6eb34246d5569@3.215.144.57:30311", "enode://82776830495703594e88b2225509a42517545ec53c3da93eea51eaa1556e3a1d56a2bd342aca5519261d40ae9ade8ff81289d7fa1a999e6e7f698aad0aa0fce6@34.236.200.111:30311", "enode://ffb8116321f8c01bc0f33852d95f8779cd2e29711c089026464907b55fdaed8d1261f49ab68f0012594633f290f005d284e4b49cb50eba7b3634f4901ec9fa7d@34.193.27.17:30311", "enode://97be405ab074c15eca1a82625f6516f08488f3c39033d72e81e98996e0f3cbc5c047e89276f953c27a08f20f41ec334e819096be543206368ae1f71c0a438b88@3.216.10.24:30311", "enode://bcdd46e1e49986d3ea31010933e7910409afcc7766dbb9335c9f27202a2361a621632a4b5a8e2bfa9bf2019a49a3658122c3e5d6e8d8a16a6984df91e7aed6c9@107.21.209.99:30311", "enode://75c5f28810dda7533bfe6dd3ba16b61c3dd2ab5198edbd8279c5e34b22615932ed826da146fdb11f62680ad238a2d961bb1c7eb6f63e2a78683584ad7e9e0103@107.20.8.154:30311"] ListenAddr = ":30311" EnableMsgEvents = false

[Node.HTTPTimeouts] ReadTimeout = 30000000000 WriteTimeout = 30000000000 IdleTimeout = 120000000000

[Node.LogConfig] FilePath = "bsc.log" MaxBytesSize = 10485760 Level = "info" FileRoot = ""

Tonyladz commented 2 years ago

Can you share your CLI cmd? … Op 26 dec. 2021 om 11:37 heeft Tonyladz @.***> het volgende geschreven: mgasps only 13，what happen and how to improve it. profile here profile_60s.txt Did you end up fixing it? My mgasps is literally around 3 at the moment and i'm running hetzner top the line dedicated root server. — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.

Sorry I'm new to this. What is cli? Did you want the bash history?

cd /home/geth/mainnet ls chown -R geth.geth systemctl start geth clear tail -f bsc.log clear sudo useradd -m geth cd /home/geth/ wget https://github.com/binance-chain/bsc/releases/download/v1.1.7/geth_linux chmod +x geth_linux nano start.sh chmod +x start.sh sudo apt-get update && sudo apt-get unzip wget https://github.com/binance-chain/bsc/releases/download/v1.1.7/mainnet.zip unzip mainnet.zip sudo apt-get install unzip unzip mainnet.zip ./geth_linux --datadir mainnet init genesis.json sudo nano /etc/systemd/system/geth.service chown -R geth.geth /home/geth/ systemctl enable geth cd mainnet rm -rf geth/* wget -O geth.tar.lz4 "https://tf-dex-prod-public-snapshot.s3-accelerate.amazonaws.com/geth-20211220.tar.lz4?AWSAccessKeyId=AKIAYINE6SBQPUZDDRRO&Signature=Bxic%2BJaL3%2BYDI8Zxf1jPgA3%2BgQ8%3D&Expires=1642605814" cd ls cd /home/ cd geth/mainnet tar -I lz4 --strip-components=2 -xvf geth.tar.lz4 /home/geth/geth_linux attach http://127.0.0.1:8545

soulcoding1994 commented 2 years ago

@Marijus their server offerings differ based on region and selected OS. Check Bare Metal > Debian 11 > Silicon Valley. Some other regions too, not all though.

@psdlt Hi can you please tell me how I can get vultr server RAId0

psdlt commented 2 years ago

@soulcoding1994 you need to create a custom iPXe script and build your server using that: Script can be as simple as this:

#!ipxe

kernel http://deb.debian.org/debian/dists/bullseye/main/installer-amd64/current/images/netboot/debian-installer/amd64/linux initrd=initrd.gz
initrd http://deb.debian.org/debian/dists/bullseye/main/installer-amd64/current/images/netboot/debian-installer/amd64/initrd.gz
boot

once server is done building, you can connect to it and install OS as if you were running it on a local computer. As one of the steps of setup you can configure your disks in whatever fashion you like (RAID0 in this case).

soulcoding1994 commented 2 years ago

@psdlt Thank you very much. I'll try !

Panjipratama837 commented 2 years ago

@guagualvcha Is the pruning done?

INFO [11-04|14:01:41.317] Pruning state data                       nodes=6,918,707,201 size=1.94TiB    elapsed=8h11m1.703s  eta=55.513s
INFO [11-04|14:01:49.318] Pruning state data                       nodes=6,920,547,567 size=1.94TiB    elapsed=8h11m9.704s  eta=47.676s
INFO [11-04|14:01:57.318] Pruning state data                       nodes=6,922,390,198 size=1.94TiB    elapsed=8h11m17.704s eta=39.822s
INFO [11-04|14:02:05.320] Pruning state data                       nodes=6,924,202,706 size=1.95TiB    elapsed=8h11m25.706s eta=32.105s
INFO [11-04|14:02:13.320] Pruning state data                       nodes=6,926,075,421 size=1.95TiB    elapsed=8h11m33.706s eta=24.125s
INFO [11-04|14:02:21.324] Pruning state data                       nodes=6,927,973,074 size=1.95TiB    elapsed=8h11m41.710s eta=16.051s
INFO [11-04|14:02:29.324] Pruning state data                       nodes=6,929,789,240 size=1.95TiB    elapsed=8h11m49.710s eta=8.311s
INFO [11-04|14:02:37.327] Pruning state data                       nodes=6,931,501,962 size=1.95TiB    elapsed=8h11m57.713s eta=1.019s
INFO [11-04|14:02:38.439] Pruned state data                        nodes=6,931,741,625 size=1.95TiB    elapsed=8h11m58.825s
INFO [11-04|14:02:41.037] Compacting database                      range=0x00-0x10 elapsed="3.329µs"

yes

how long does it take to finish?

vae520283995 commented 2 years ago

how long does it take to finish?

Don't prune, download the latest snapshot

cpecorari commented 2 years ago

how long does it take to finish?

Don't prune, download the latest snapshot

For me it is faster to prune, like 4 hours to prune, then about same time (4h) to keep up to sync again. Latest snapshot + sync is about 2 days... -> On baremetal server 8 cores / 16 threads and 64Go RAM, 2To SSD M2 NVme.

nlmtri commented 2 years ago

What does --cache mean? I have 128 GB RAM, should I set --cache 100000 ?

RickSanchez1974 commented 2 years ago

@Lajoix @jcaffet folks, what kind of filing systems do you use on your servers? I've read somewhere that xfs is better than ext4 (sorry, don't remember where; don't have a link). Also, do you use single disk or RAID? I have one server on AWS (i3en.2xlarge, RAID0, xfs), been running it for ~half a year, never had issues you're describing. Few days ago I've setup another server on Vultr (also RAID0, also xfs) - it fast synced to latest block from scratch in under a day.

If you're constantly having sync issues and can't catch up to network - review your hardware setup, maybe spin up a different instance on a different region, maybe you just got a busy host, who knows.

Might be a little late, but how do you configure your instance to run in Raid 0?

mj-dcb commented 2 years ago

75% of total RAM would be safe

Op 11 jan. 2022 om 17:57 heeft Nguyen Le Minh Tri - GCS200185 @.***> het volgende geschreven:

What does --cache mean? I have 128 GB RAM, should I set --cache 100000 ?

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.

bert2002 commented 2 years ago

Hi, we are trying to run a full node (archive + tracing) and pruning does not work when tracing is enabled (correct me if I am wrong). Our machine is constantly 2 month behind HEAD and I am wondering how this can be. The specifications are:

AWS Instance Type: c5.4xlarge (16 core and 32gb of memory) Disk: gp3 with 6TB, 16000 iops and throughput of 500mb

Anyone running into similar problem with such machine?

seonggwonyoon commented 2 years ago

Hi, we are trying to run a full node (archive + tracing) and pruning does not work when tracing is enabled (correct me if I am wrong). Our machine is constantly 2 month behind HEAD and I am wondering how this can be. The specifications are:

AWS Instance Type: c5.4xlarge (16 core and 32gb of memory) Disk: gp3 with 6TB, 16000 iops and throughput of 500mb

Anyone running into similar problem with such machine?

I have experienced such a problem several times, so I changed the specification from c5.4xlarge to i3en.2xlarge 5 months ago, downloaded the latest snapshot and synced it, and I am using it without any problems until now.

bert2002 commented 2 years ago

I have experienced such a problem several times, so I changed the specification from c5.4xlarge to i3en.2xlarge 5 months ago, downloaded the latest snapshot and synced it, and I am using it without any problems until now.

Do you use EBS volume for storage or the attached NVMe's? (because they are only 2x2500mb) and my node is already at ~4.9TB.

seonggwonyoon commented 2 years ago

I have experienced such a problem several times, so I changed the specification from c5.4xlarge to i3en.2xlarge 5 months ago, downloaded the latest snapshot and synced it, and I am using it without any problems until now.

Do you use EBS volume for storage or the attached NVMe's? (because they are only 2x2500mb) and my node is already at ~4.9TB.

I'm using attached nvme disk. If the data size of the node is large, it is recommended to delete it, download a new snapshot, and synchronize it.

bert2002 commented 2 years ago

I'm using attached nvme disk. If the data size of the node is large, it is recommended to delete it, download a new snapshot, and synchronize it.

I dont think this works for tracing enabled nodes?

ofarukcaki commented 2 years ago

Is it safe to prune state before fully downloading all known states?

Edit: looks like it doesn't allow me to do it. Is there any way? I have ~100gb free space left from my 1TB storage and I only downloaded 1/5 of the known states

acswap commented 2 years ago

Just download the data from the authorities.

通过 ProtonMail 安全邮箱发送。

------- Original Message ------- 2022年5月11日星期三 11:04 上午，ပြည့်ဖြိုးအောင် @.***> 来信：

Please continue to help.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

zyhyung commented 2 years ago

Hi guys,

Is the pruning command line still the same? According to this release, there is a new data prune tool introduced. https://github.com/bnb-chain/bsc/releases/tag/v1.1.8

I am trying to prune on my BSC full node but i'm unsure of the command.

Any help will be appreciated. Thanks!

michaelr524 commented 2 years ago

We've successfully started a full node from snapshot. The snapshot started at 1.5T - is this the minimal size at this time? BTW, tried pruning and get this error: head doesn't match snapshot: have 0xdbdcc7285975a1077c86cc1f1913fc490c4a85817b729fdacafdac4da6953b6b, want 0x919fcc7ad870b53db0aa76eb588da06bacb6d230195100699fc928511003b422 Anyone knows what does this mean?

du5 commented 2 years ago

We've successfully started a full node from snapshot. The snapshot started at 1.5T - is this the minimal size at this time? BTW, tried pruning and get this error: head doesn't match snapshot: have 0xdbdcc7285975a1077c86cc1f1913fc490c4a85817b729fdacafdac4da6953b6b, want 0x919fcc7ad870b53db0aa76eb588da06bacb6d230195100699fc928511003b422 Anyone knows what does this mean?

BNB48Club/bsc-snapshots contributing another dump of snapshot (543.61G)

michaelr524 commented 2 years ago

Cool! How do you make these? Could you share any links to docs or explanation? @du5

ofarukcaki commented 2 years ago

We've successfully started a full node from snapshot. The snapshot started at 1.5T - is this the minimal size at this time? BTW, tried pruning and get this error: head doesn't match snapshot: have 0xdbdcc7285975a1077c86cc1f1913fc490c4a85817b729fdacafdac4da6953b6b, want 0x919fcc7ad870b53db0aa76eb588da06bacb6d230195100699fc928511003b422 Anyone knows what does this mean?

BNB48Club/bsc-snapshots contributing another dump of snapshot (543.61G)

How it can be that much small? I have pruned my node and it only reduced to 1.3TB

du5 commented 2 years ago

follow the official documentation

tpalaz commented 2 years ago

I've successfully setup and ran my fullnode using BNB48Club's snapshots and have been able to get it to sync. One issue I'm facing is that historic transactions aren't able to be retreived via RPC calls. Specifically, eth_getTransaction and getTransactionReceipt return null. Whereas if the transaction was recent (within a few blocks ago), the node has no problem serving it.

Is this due to the snapshot that was downloaded? Or could it be configuration of the node? For reference, my startup parameters are:

--config ./config.toml --datadir ./mainnet --cache 32000 --rpc.allow-unprotected-txs --txlookuplimit 0 --http --maxpeers 100 --ws --syncmode=full --snapshot=true --diffsync" > /home/geth/start.sh

IOPS and mgasps are exceeding (averaging around 300 mgasps - so not synchronization is not the issue).

Likewise, my config file is here:

[Unit]
Description=BSC Full Node

[Service]
User=geth
Type=simple
WorkingDirectory=/home/geth
ExecStart=/bin/bash /home/geth/start.sh
Restart=on-failure
RestartSec=30
TimeoutSec=300
IOWeight=8000
CPUWeight=8000

[Install]
WantedBy=default.target

Any advice on how to fix this issue? The only possible thing I could think of was the snapshot not having the correct indexes - which doesn't make a ton of sense and probably wouldn't even allow sync.

du5 commented 2 years ago

@tpalaz the reason for the small size is that the historical transactions are prune

du5 commented 2 years ago

@tpalaz If the transaction you need to retrieve is a long time past, it is recommended to use erigon and turn off prune, it will surprise you with its efficiency

tpalaz commented 2 years ago

@tpalaz the reason for the small size is that the historical transactions are prune

@du5 I guess that's why the download is so small. Does BNB48's erigon snapshot prune as well? Or would you recommend syncing from 0 :(

bnb-chain / bsc