Closed kaber2 closed 1 year ago
We are one of the BSC validator nodes and provide public API services to the BSC community. As we have observed, the volume of transactions on the BSC chain has reached an unprecedented level. We want to share some pointers to help speed up syncing on your full node:
1 Use the latest binary version v1.1.5 and add the --diffsync
flag when starting the node
2 Recommended configuration (on AWS): at least c5a.8xlarge with a gp3 disk configured for 12000 IOPS or higher
3 Use the latest official snapshot: https://github.com/binance-chain/bsc-snapshots
4 Configure your node based on the latest config.toml:
https://github.com/binance-chain/bsc/releases/download/v1.1.5/mainnet.zip
5 Try temporarily turn off the RPC port to speed up synchronization
If you have more problems with node synchronization, please feel free to utilize our platform (https://infstones.com/) to create a free BSC Public API project or create your dedicated node with only one click.
I have two BSC full node. In every month, I will do db prune. Now my two BSC nodes work well. Do you guys have tried prune db? Besides, according to my experience, AWS Ireland machine would have better connection with other BSC p2p nodes.
The challenges that node operators would encounter come as no surprise, considering the excellent user experience for transactions and DeFi on BSC: extremely low fees & extremely fast speed of transaction confirmation. For the normal use cases, using the APIs provided by professional service providers is good enough to satisfy all the needs.
My node can finish the synchronization process within 2 days, as long as the the following requirements are met: a) Use binary v1.1.5 b) Use --diffsync, which can improve the syncing speed. c) RPC port turned off you can find the latest data released by BSC team here: https://github.com/binance-chain/bsc-snapshots . Hope it helps
what are the incentives to running a BSC full node? Any references?
what are the incentives to running a BSC full node? Any references?
unlimited "chain analysis"
Maintaining the BSC node is a terrible experience, it wastes too much of my time
My node can finish the synchronization process within 2 days, as long as the the following requirements are met: a) Use binary v1.1.5 b) Use --diffsync, which can improve the syncing speed. c) RPC port turned off you can find the latest data released by BSC team here: https://github.com/binance-chain/bsc-snapshots . Hope it helps
How to turn of RPC port?
I have 64RAM, 2TB NVME server. And i have no issues (besides block forks). Synced from snapshot, works like a charm. Im re-initializing node from snapshot once a month, it takes a day, and I have no problems.
I have 64RAM, 2TB NVME server. And i have no issues (besides block forks). Synced from snapshot, works like a charm. Im re-initializing node from snapshot once a month, it takes a day, and I have no problems.
Your config?
I have 64RAM, 2TB NVME server. And i have no issues (besides block forks). Synced from snapshot, works like a charm. Im re-initializing node from snapshot once a month, it takes a day, and I have no problems.
Your config?
Just recommended config from binance's fullnode guide
It is reasonable to use high configuration to run the nodes since the amount of daily transactions on the BSC chain has reached an incredible 15 million. And the peak TPS has probably already reached 300-400. All users are gathering into the BSC ecosystem.
My issue #539 was fixed after upgrading to 1.1.15. Always up to date without any lag spikes.
Just move over to Expanse, the first stable clone of Ethereum. They have active development, just removed the DAG, less than $1million mcap.... so much better upside and potential with the American iteration launched in 2015, chain id=2
My issue #539 was fixed after upgrading to 1.1.15. Always up to date without any lag spikes.
your machine spec: 96 vcpus, 384 GiB memory
Ok then, it is too expensive for me
Hi, this is Stanley at Ankr, which is a web3 infrastructure company hosting one of binance validator and also providing both RPC and staking service to the public. The following are tricks we utilize at Ankr.
If you are considering a 3rd party RPC service provider, free free to apply one from https://app.ankr.com. In the meanwhile, Ankr will provide distributed RPC solution for BSC to the public soon. Stay tuned.
My issue #539 was fixed after upgrading to 1.1.15. Always up to date without any lag spikes.
your machine spec: 96 vcpus, 384 GiB memory
Ok then, it is too expensive for me
Yes it is one of the most expensive options available (3600 euro per month), but it seems to be an overkill anyway. Maximum observed disk usage is ~40%. I will try with lower cost options with the same version, configuration & location and see if it can still keep up.
We created a Telegram group for BSC Node discussions, feel free to join and share experience https://t.me/joinchat/zXCza2tQN0tjYzM0
yes, invite me again, please!
I wonder why smart people even waste time with this crap? Initial poster @kaber2 seems very knowledgeable. I wonder why touch anything BSC at all in the first place? Just delete all software, sell BNB and spend your time on projects that care about science and not quick money grabs and marketing (on shoulders of others).
BSC is quite a disgusting project, from the start, it was very obvious. Now the moment has come, it's fun for some. I really enjoy reading this thread. Maybe this was the real value of BSC ecosystem... to confirm the obvious predictions of smart people that actually think about what they are doing with code, economics, decentralized designs, bare basics of computer science and all the rest that is needed, for example Ethereum developers and some others. Maybe learn to listen to such voices, it helps grow your brain.
Spending time with BSC and similar projects makes you dumber and not so much richer at the end.
Also related, from issue https://github.com/binance-chain/bsc/issues/338 it's demential to read that the recommended way to keep your node synced is restoring snapshots every 3 days.
The upcoming hard fork on 30th November is only going to make things worse. The health of the network is terrible and the proposed solution to address nodes issues is burning tx fees and pumping BNB, nice! :+1:
In the BSC Daily Transactions Chart here you can clearly see where is the limit of the network (~ 12,5M txs). During the last weeks there have been 3 surprisingly similar transactions peaks on 29th October, 3rd November and 11th November. Looks like the limit was reached, nodes got unsynced, less txs came in and after some days the network recovered its health. I operate a BSC node and found that in every peak my overprovisioned node unsynced. The peak on 16th November really surprised me, have validators made some upgrades on the infra?
During some days, even bscscan has been outdated for hours...
(written by a 0 BNB holder, never considered buying this garbage)
check https://github.com/tharsis/evmos
This thread is crazy, as I see this from the point of view of a Polygon validator node.
As validators we have dedicated channels on discord to discuss everything about nodes maintenance and upgrades, and we also have a dedicated team in Polygon to monitor nodes and help.
How is this possible that Binance doesn't care while having a network ranked #3?
You should ask Binance to take the matter seriously and set an ultimatum. Node operators aren't that easy to replace.
Recommended related reading is Vitalik's post on "The Limits to Blockchain Scalability": https://vitalik.ca/general/2021/05/23/scaling.html
We are one of the BSC validator nodes and provide public API services to the BSC community. As we have observed, the volume of transactions on the BSC chain has reached an unprecedented level. We want to share some pointers to help speed up syncing on your full node:
1 Use the latest binary version v1.1.5 and add the
--diffsync
flag when starting the node 2 Recommended configuration (on AWS): at least c5a.8xlarge with a gp3 disk configured for 12000 IOPS or higher 3 Use the latest official snapshot: https://github.com/binance-chain/bsc-snapshots 4 Configure your node based on the latest config.toml: https://github.com/binance-chain/bsc/releases/download/v1.1.5/mainnet.zip 5 Try temporarily turn off the RPC port to speed up synchronizationIf you have more problems with node synchronization, please feel free to utilize our platform (https://infstones.com/) to create a free BSC Public API project or create your dedicated node with only one click.
Bro, are you kidding? Nothing will work on c5a.8xlarge since only the cloud disk is supported there. You won't be able to synch a node there even from a snapshot that appeared an hour ago. And 12k or 16k iops won't help you in any way (I checked it personally myself 2 days ago) . Could you just write us the exact technical parameters of the server that is used for your validator node?
Maybe time for BSC to become a volition and settle on Ethereum. They they won't have to worry about L1 security — and the inevitable (complete) centralization of BSC will be less of an issue because there will always be an escape hatch to a true L1.
Hi, this is Stanley at Ankr, which is a web3 infrastructure company hosting one of binance validator and also providing both RPC and staking service to the public. The following are tricks we utilize at Ankr.
- We are majorly using bare metals to achieve the best performance given the same level of hardware among major cloud providers.
- CPU: AMD Ryzen 5950x is good enough to power both full and archive mode
- Memory: DDR4 ECC 128 GB for full, 256 GB for archive. No matter you are using commerical or comsumer based hardware, please make sure using a ECC type of memory!!!
- Storage: This is a tricky one. Local NVME would dramatically improve the performance of BSC. So far archive mode is requiring 16TB data and therefore, please leave 30TB for the foreseeable expansion.
- Network: This is not critical, a regular 100MB public bandwidth is good enough for syncing from scratch.
- Feel free to use any cloud-based VM match the above specs. But according to our experience, bare metals are the ONLY winner for a RPC service provider in terms of cost and performance.
If you are considering a 3rd party RPC service provider, free free to apply one from app.ankr.com. In the meanwhile, Ankr will provide distributed RPC solution for BSC to the public soon. Stay tuned.
To add to @fruit37 comments:
sudo mkfs.ext4 -F -E stripe-width=256,lazy_itable_init=0,lazy_journal_init=0,discard /dev/md0
rite cache flushes are fairly slow on local SSDs. You can increase the write performance for some apps by disabling automatic flush commands in those apps or by disabling flush options at the file system level.
To disable write cache flushing on ext4
file systems, include the nobarrier
setting in your mount options or in your /etc/fstab entries. For example:
sudo mount -o discard,defaults,nobarrier /dev/[LOCAL_SSD_ID] /mnt/disks/[MNT_DIR]
where: [LOCAL_SSD_ID] is the device ID for the local SSD that you want to mount and [MNT_DIR] is the directory in which to mount it.
blk-mq
see https://yarondar.wordpress.com/2018/07/29/have-you-tried-blk-mq/
Also to hijack @TimDaub comment, see this especially pertinent post by Vitalik: toward-a-12-second-block-time
TLDR: use OVH/Equinix/Hive/etc
To get maximal throughput it is critical to use optimal TCP send and receive socket buffer sizes for the link you are using. If the buffers are too small, the TCP congestion window will never fully open up. If the receiver buffers are too large, TCP flow control breaks and the sender can overrun the receiver, which will cause the TCP window to shut down. This is likely to happen if the sending host is faster than the receiving host (re: this can cause consensus issues or exacerbate existing issues, e.g. out of turn blocks for Clique POA). (lets ignore for a moment why TCP isn't even what you should ideally be using for now)
This is by no means comprehensive but I think it touches on several points that have been outlined here.
Hi, this is Stanley at Ankr, which is a web3 infrastructure company hosting one of binance validator and also providing both RPC and staking service to the public. The following are tricks we utilize at Ankr.
- We are majorly using bare metals to achieve the best performance given the same level of hardware among major cloud providers.
- CPU: AMD Ryzen 5950x is good enough to power both full and archive mode
- Memory: DDR4 ECC 128 GB for full, 256 GB for archive. No matter you are using commerical or comsumer based hardware, please make sure using a ECC type of memory!!!
- Storage: This is a tricky one. Local NVME would dramatically improve the performance of BSC. So far archive mode is requiring 16TB data and therefore, please leave 30TB for the foreseeable expansion.
- Network: This is not critical, a regular 100MB public bandwidth is good enough for syncing from scratch.
- Feel free to use any cloud-based VM match the above specs. But according to our experience, bare metals are the ONLY winner for a RPC service provider in terms of cost and performance.
If you are considering a 3rd party RPC service provider, free free to apply one from https://app.ankr.com. In the meanwhile, Ankr will provide distributed RPC solution for BSC to the public soon. Stay tuned.
Just tried this service for fun. the same delay of 15-30 seconds.
Just tried this service for fun. the same delay of 15-30 seconds.
@edgeofthegame just curious, how are you quantifying this delay?
Just tried this service for fun. the same delay of 15-30 seconds.
@edgeofthegame just curious, how are you quantifying this delay?
If you create an API in Ankr it says this, looks like it desyncs a bit from time to time:
It's crazy, how they are still not responding officially to this issue. No "Member" or "Contributor" in this discussion right now.
Hi, this is Stanley at Ankr, which is a web3 infrastructure company hosting one of binance validator and also providing both RPC and staking service to the public. The following are tricks we utilize at Ankr.
- We are majorly using bare metals to achieve the best performance given the same level of hardware among major cloud providers.
- CPU: AMD Ryzen 5950x is good enough to power both full and archive mode
- Memory: DDR4 ECC 128 GB for full, 256 GB for archive. No matter you are using commerical or comsumer based hardware, please make sure using a ECC type of memory!!!
- Storage: This is a tricky one. Local NVME would dramatically improve the performance of BSC. So far archive mode is requiring 16TB data and therefore, please leave 30TB for the foreseeable expansion.
- Network: This is not critical, a regular 100MB public bandwidth is good enough for syncing from scratch.
- Feel free to use any cloud-based VM match the above specs. But according to our experience, bare metals are the ONLY winner for a RPC service provider in terms of cost and performance.
If you are considering a 3rd party RPC service provider, free free to apply one from https://app.ankr.com. In the meanwhile, Ankr will provide distributed RPC solution for BSC to the public soon. Stay tuned.
128GB? I currently have 64GB bare metal system and regardless what I set, the node will never use more than 32.4GB of ram (commited max is~40GB). How can you effectively use 100+ GB? I'd like to know and use the full hardware.
@zhongfu last accessible block timestamp minus current time (checking a few times per sec). usually it was 1-2 seconds. now it's 10-20 seconds. @bostarch don't bother they have the same lags as everybody else, just tested
Wow guys, reading all these comments really hurts. Why does nobody care about the chain, the network and the people who are running the nodes?!?!!?!? Should be one of the main priorities.
It's also insane to read how immense the hardware requirements are for running a node. It's worse than a SOL node :laughing:
If you are looking for a top team that is building a blockchain with upgradeable smart contracts, a microservice based node, fee less transactions, support for multi languages and many other innovative things... Read about Koinos. Try to set up a node for test net. The only thing you need to do is to start a docker-compose script! Check https://github.com/koinos/koinos or koinos.io
I'm not a coder nor do I run a node, I've just been doing my research on where to set up my project for the working class and this is a prime example of what's wrong in the world. Here's a small part quoted from one of my posts I made and why we should change those things mentioned in this repo
I have absolutely nothing against people with a billion $. It’s how many of them, got it and how they use it, that the problem is. Their power and greed gained on the back of the working class people, being used and thrown away as garbage, while governments (put in place by the wealthiest) hope that they die as soon as they get their first pensioncheque. It’s time for wealth sharing according to the capability of a person to do something and what that’s worth to others. What is the value to society of the builders, the person that cleans operating rooms with an eye for detail, factoryworkers making the tools, a surgeon, a media spokesperson of a hospital? Who needs who and do we need that politician who decides that a hospital can be build in that city? Or can people vote on that decision that there should be a hospital there? I doubt anyone in the world has a problem with people with talent, knowledge and skills to be paid accordingly but is that the reality?
Use your time and talents for the people who care! The whole post is here if you agree.
I already had my doubts about the intentions of Binance now I'm really sure I won't be building on their chain either. They just want to be a bank who doesn't care about their workers and customers, like so many others in the current system.
Bro, are you kidding? Nothing will work on c5a.8xlarge since only the cloud disk is supported there. You won't be able to synch a node there even from a snapshot that appeared an hour ago. And 12k or 16k iops won't help you in any way (I checked it personally myself 2 days ago) . Could you just write us the exact technical parameters of the server that is used for your validator node?
Hey @Zer34, not sure why it didn't work in your case. I just launched another node using the exact specs mentioned; the node was synced and ran stably. Would you mind letting me know if you tried all the listed items at once (latest binary, 'diffsync', latest config, RPC port off, etc.)? I found it crucial to do all on the node for it to work. If you still have synchronization issues, I suggest using an i3.4xlarge or a better instance to get synced faster.
I am one of those who commit many Pull Requests in the BSC repo. We are keeping our heads down on building and optimizing the BSC to meet the requirements from 2M daily active addresses (ATH recently)
I apologize that we did not put more effort to improve enough transparency and communication for each release.
However, we do care a lot for the value of the community, and really appreciated your critizations and suggestions. We will work with our supportive community to keep improving the stability of infrastructure and introduce new scalable solutions.
Let me have some summary here:
We'll keep utilizing Github as a builder collaboration platform to raise the proposal, create concrete issues with facts (e.g., logs), discuss solutions, and build infra. For other types of discussion, join our public communities and feel free to ping our admins whenever necessary.
Node discussion discord: https://discord.gg/gx9E5fCvmW Dev discussion discord: https://discord.gg/E6yDKuV5MT
Thank you for your patience and support!
Come on Build on Bitcoin. The future is there!
why would anyone want to build on bitcoin with 10 min block times?
I've switched to Solana
Bro, are you kidding? Nothing will work on c5a.8xlarge since only the cloud disk is supported there. You won't be able to synch a node there even from a snapshot that appeared an hour ago. And 12k or 16k iops won't help you in any way (I checked it personally myself 2 days ago) . Could you just write us the exact technical parameters of the server that is used for your validator node?
Hey @Zer34, not sure why it didn't work in your case. I just launched another node using the exact specs mentioned; the node was synced and ran stably. Would you mind letting me know if you tried all the listed items at once (latest binary, 'diffsync', latest config, RPC port off, etc.)? I found it crucial to do all on the node for it to work. If you still have synchronization issues, I suggest using an i3.4xlarge or a better instance to get synced faster.
@rudy-infstones Can you give your instance type, system configurations, config.toml, all commands you execute till the node is syncing? I want to follow your steps strictly. Thanks.
Binance Smart Chain Binance Chain I would like to know how much of my assets I have now and how much farming investment I have. I know the details, what should I do? Thank you very much.
We are one of the BSC validator nodes and provide public API services to the BSC community. As we have observed, the volume of transactions on the BSC chain has reached an unprecedented level. We want to share some pointers to help speed up syncing on your full node:
1 Use the latest binary version v1.1.5 and add the
--diffsync
flag when starting the node 2 Recommended configuration (on AWS): at least c5a.8xlarge with a gp3 disk configured for 12000 IOPS or higher 3 Use the latest official snapshot: https://github.com/binance-chain/bsc-snapshots 4 Configure your node based on the latest config.toml: https://github.com/binance-chain/bsc/releases/download/v1.1.5/mainnet.zip 5 Try temporarily turn off the RPC port to speed up synchronizationIf you have more problems with node synchronization, please feel free to utilize our platform (https://infstones.com/) to create a free BSC Public API project or create your dedicated node with only one click.
Bro, are you kidding? Nothing will work on c5a.8xlarge since only the cloud disk is supported there. You won't be able to synch a node there even from a snapshot that appeared an hour ago. And 12k or 16k iops won't help you in any way (I checked it personally myself 2 days ago) . Could you just write us the exact technical parameters of the server that is used for your validator node?
Hey @Zer34, not sure why it didn't work in your case. I just launched another node using the exact specs mentioned; the node was synced and ran stably. Would you mind letting me know if you tried all the listed items at once (latest binary, 'diffsync', latest config, RPC port off, etc.)? I found it crucial to do all on the node for it to work. If you still have synchronization issues, I suggest using an i3.4xlarge or a better instance to get synced faster.
@rudy-infstones Can you give your instance type, system configurations, config.toml, all commands you execute till the node is syncing? I want to follow your steps strictly. Thanks.
Hey @hellodword , please reference my previous comment regarding the setup. We used c5a.8xlarge and the exact official config.toml. Feel free to let me know if you have any further questions. You can also try to set up a BSC node on our platform (https://cloud.infstones.io/)
guys despite all the problems the network has currently it reached huge milestones, the fact that people only complain and dont try to come up with practical and implementable solutions makes me sad, even if you are not a coder or programmer, try at least to read and understand the basics of blockchain technology and put in some effort to think what might be the problem for your current situation and talk publicly providing insights and useful information that other people can learn from, only complaining wont speed up the sync and make the network better thats for sure.
let this moment be a uniting moment for all of us, when a new phase of bsc begins.
here for the memez.
@edgeofthegame jokes aside, once my server's disk died from running 2 nodes at the same time and the hosting provider had to replace it
At C.R.E.A.M. we’ve been running our BSC validator node on AWS EC2 m5zn.3xlarge with 2.5 TB gp3 SSD 3000 IOPS 125 MB/s and it’s been very stable for a long time. We started seeing sync issues starting November 3rd and started missing blocks then. Last week we enabled the new diff sync feature from BSC and upgraded the disk performance to 8000 IOPS, 250MB/s and our node has been stable ever since. We recommend others to try the same. Like any cutting edge technology, there are often challenges in keeping up with the latest and greatest. While this thread and discussion has spawned many valid concerns, we believe this is a healthy discussion around how BSC can improve moving forward. We agree with the need for a shared channel where validators can learn from one another and we’re glad to see BSC has already deployed this. Looking forward to more innovations to come!
BSC, and its issues, stems from GoEthereum that is 2014 tech. BSC can improve, but only if they invest in building a more open and healthy community. For the sync issues, they are very real and I wrote a Twitter thread here:
https://twitter.com/moo9000/status/1463454127095042050
(Contains some useful tips for those who are struggling with syncing.)
Just done syncing a new node on Micron SATA drive, it took less than 14 hours. So there is no need for NVMe or whatever. SATA bandwidth is more than enough (20x more than needed actually) and latency is not much of an issue as it seemed to be.
@CryptoManiac no need, yes, but it's really really not recommended -- you'd very likely see it falling behind quite often
what's the mgasps
like on your node anyway
@CryptoManiac no need, yes, but it's really really not recommended -- you'd very likely see it falling behind quite often
Well I can say that NVMe wouldn't be of much help anyway. I spent a week testing and comparing various hardware and came to this exact realization that enterprise-grade SATA drive is more than enough.
what's the
mgasps
like on your node anyway
I sometimes see 500-700 mgasps but it's usually between 60 and 200. It's like a dice. :)
Ubuntu Linux 20.04 on 8-core Xeon and 32 Gb RAM. I tried to install 64 Gb and check the results and can say that it wouldn't hurt but rather pointless.
No tweaks or whatever, everything is literally running at default settings including the BSC client itself. Perhaps the main difference from other reports is that it's not a virtual machine but a physical hardware.
P.S. I forgot to mention that system and the BSC data folder are located on Micron 5300 Pro 3.84Tb MTFDDAK3T8TDS.
Guys, seriously, WTF. This is a blockchain with supposedly billions of value, yet it is governed and developed like the project of a stoned teenager.
I've rarely seen something handled so unprofessionally.
Overall, there is only one conclusion. Binance wanted a quick hack to make some money, but is not willing to expand even modest resources to make this thing actually work. Given that they've made billions from this, this is absurd and a huge abuse of the trust (and money) people put in this.
You have proven to be incompetent of leading, developing and governing this. Just be honest and trash it without more people wasting their time and money.
(written by a 4k BNB holder, considering dumping this garbage)