bnb-chain / bsc

A BNB Smart Chain client based on the go-ethereum fork
GNU Lesser General Public License v3.0
2.73k stars 1.56k forks source link

BSC is a lost cause #553

Closed kaber2 closed 1 year ago

kaber2 commented 3 years ago

Guys, seriously, WTF. This is a blockchain with supposedly billions of value, yet it is governed and developed like the project of a stoned teenager.

I've rarely seen something handled so unprofessionally.

Overall, there is only one conclusion. Binance wanted a quick hack to make some money, but is not willing to expand even modest resources to make this thing actually work. Given that they've made billions from this, this is absurd and a huge abuse of the trust (and money) people put in this.

You have proven to be incompetent of leading, developing and governing this. Just be honest and trash it without more people wasting their time and money.

(written by a 4k BNB holder, considering dumping this garbage)

rudy-infstones commented 3 years ago

We are one of the BSC validator nodes and provide public API services to the BSC community. As we have observed, the volume of transactions on the BSC chain has reached an unprecedented level. We want to share some pointers to help speed up syncing on your full node:

1 Use the latest binary version v1.1.5 and add the --diffsync flag when starting the node 2 Recommended configuration (on AWS): at least c5a.8xlarge with a gp3 disk configured for 12000 IOPS or higher 3 Use the latest official snapshot: https://github.com/binance-chain/bsc-snapshots 4 Configure your node based on the latest config.toml: https://github.com/binance-chain/bsc/releases/download/v1.1.5/mainnet.zip 5 Try temporarily turn off the RPC port to speed up synchronization

If you have more problems with node synchronization, please feel free to utilize our platform (https://infstones.com/) to create a free BSC Public API project or create your dedicated node with only one click.

braydenbc commented 3 years ago

I have two BSC full node. In every month, I will do db prune. Now my two BSC nodes work well. Do you guys have tried prune db? Besides, according to my experience, AWS Ireland machine would have better connection with other BSC p2p nodes.

jasperjiang93 commented 3 years ago

The challenges that node operators would encounter come as no surprise, considering the excellent user experience for transactions and DeFi on BSC: extremely low fees & extremely fast speed of transaction confirmation. For the normal use cases, using the APIs provided by professional service providers is good enough to satisfy all the needs.

nyuspc commented 3 years ago

My node can finish the synchronization process within 2 days, as long as the the following requirements are met: a) Use binary v1.1.5 b) Use --diffsync, which can improve the syncing speed. c) RPC port turned off you can find the latest data released by BSC team here: https://github.com/binance-chain/bsc-snapshots . Hope it helps

yashdesai87 commented 3 years ago

what are the incentives to running a BSC full node? Any references?

xpkore commented 3 years ago

what are the incentives to running a BSC full node? Any references?

unlimited "chain analysis"

ghost commented 3 years ago

Maintaining the BSC node is a terrible experience, it wastes too much of my time

xcuocao commented 3 years ago

My node can finish the synchronization process within 2 days, as long as the the following requirements are met: a) Use binary v1.1.5 b) Use --diffsync, which can improve the syncing speed. c) RPC port turned off you can find the latest data released by BSC team here: https://github.com/binance-chain/bsc-snapshots . Hope it helps

How to turn of RPC port?

apepenkov commented 3 years ago

I have 64RAM, 2TB NVME server. And i have no issues (besides block forks). Synced from snapshot, works like a charm. Im re-initializing node from snapshot once a month, it takes a day, and I have no problems.

xcuocao commented 3 years ago

I have 64RAM, 2TB NVME server. And i have no issues (besides block forks). Synced from snapshot, works like a charm. Im re-initializing node from snapshot once a month, it takes a day, and I have no problems.

Your config?

apepenkov commented 3 years ago

I have 64RAM, 2TB NVME server. And i have no issues (besides block forks). Synced from snapshot, works like a charm. Im re-initializing node from snapshot once a month, it takes a day, and I have no problems.

Your config?

Just recommended config from binance's fullnode guide

ZivLi commented 3 years ago

It is reasonable to use high configuration to run the nodes since the amount of daily transactions on the BSC chain has reached an incredible 15 million. And the peak TPS has probably already reached 300-400. All users are gathering into the BSC ecosystem.

NullQubit commented 3 years ago

My issue #539 was fixed after upgrading to 1.1.15. Always up to date without any lag spikes.

SmartContractX commented 3 years ago

Just move over to Expanse, the first stable clone of Ethereum. They have active development, just removed the DAG, less than $1million mcap.... so much better upside and potential with the American iteration launched in 2015, chain id=2

Https://wagmi.club

ghost commented 3 years ago

My issue #539 was fixed after upgrading to 1.1.15. Always up to date without any lag spikes.

your machine spec: 96 vcpus, 384 GiB memory

Ok then, it is too expensive for me

fruit37 commented 3 years ago

Hi, this is Stanley at Ankr, which is a web3 infrastructure company hosting one of binance validator and also providing both RPC and staking service to the public. The following are tricks we utilize at Ankr.

  1. We are majorly using bare metals to achieve the best performance given the same level of hardware among major cloud providers.
  2. CPU: AMD Ryzen 5950x is good enough to power both full and archive mode
  3. Memory: DDR4 ECC 128 GB for full, 256 GB for archive. No matter you are using commerical or comsumer based hardware, please make sure using a ECC type of memory!!!
  4. Storage: This is a tricky one. Local NVME would dramatically improve the performance of BSC. So far archive mode is requiring 16TB data and therefore, please leave 30TB for the foreseeable expansion.
  5. Network: This is not critical, a regular 100MB public bandwidth is good enough for syncing from scratch.
  6. Feel free to use any cloud-based VM match the above specs. But according to our experience, bare metals are the ONLY winner for a RPC service provider in terms of cost and performance.

If you are considering a 3rd party RPC service provider, free free to apply one from https://app.ankr.com. In the meanwhile, Ankr will provide distributed RPC solution for BSC to the public soon. Stay tuned.

NullQubit commented 3 years ago

My issue #539 was fixed after upgrading to 1.1.15. Always up to date without any lag spikes.

your machine spec: 96 vcpus, 384 GiB memory

Ok then, it is too expensive for me

Yes it is one of the most expensive options available (3600 euro per month), but it seems to be an overkill anyway. Maximum observed disk usage is ~40%. I will try with lower cost options with the same version, configuration & location and see if it can still keep up.

Sini0r commented 3 years ago

We created a Telegram group for BSC Node discussions, feel free to join and share experience https://t.me/joinchat/zXCza2tQN0tjYzM0

yes, invite me again, please!

davidhq commented 3 years ago

I wonder why smart people even waste time with this crap? Initial poster @kaber2 seems very knowledgeable. I wonder why touch anything BSC at all in the first place? Just delete all software, sell BNB and spend your time on projects that care about science and not quick money grabs and marketing (on shoulders of others).

BSC is quite a disgusting project, from the start, it was very obvious. Now the moment has come, it's fun for some. I really enjoy reading this thread. Maybe this was the real value of BSC ecosystem... to confirm the obvious predictions of smart people that actually think about what they are doing with code, economics, decentralized designs, bare basics of computer science and all the rest that is needed, for example Ethereum developers and some others. Maybe learn to listen to such voices, it helps grow your brain.

Spending time with BSC and similar projects makes you dumber and not so much richer at the end.

PabloCastellano commented 3 years ago

Also related, from issue https://github.com/binance-chain/bsc/issues/338 it's demential to read that the recommended way to keep your node synced is restoring snapshots every 3 days.

The upcoming hard fork on 30th November is only going to make things worse. The health of the network is terrible and the proposed solution to address nodes issues is burning tx fees and pumping BNB, nice! :+1:

In the BSC Daily Transactions Chart here you can clearly see where is the limit of the network (~ 12,5M txs). During the last weeks there have been 3 surprisingly similar transactions peaks on 29th October, 3rd November and 11th November. Looks like the limit was reached, nodes got unsynced, less txs came in and after some days the network recovered its health. I operate a BSC node and found that in every peak my overprovisioned node unsynced. The peak on 16th November really surprised me, have validators made some upgrades on the infra?

imagen

During some days, even bscscan has been outdated for hours...

(written by a 0 BNB holder, never considered buying this garbage)

fedekunze commented 3 years ago

check https://github.com/tharsis/evmos

https://twitter.com/EvmosOrg

ffmad commented 3 years ago

This thread is crazy, as I see this from the point of view of a Polygon validator node.

As validators we have dedicated channels on discord to discuss everything about nodes maintenance and upgrades, and we also have a dedicated team in Polygon to monitor nodes and help.

How is this possible that Binance doesn't care while having a network ranked #3?

You should ask Binance to take the matter seriously and set an ultimatum. Node operators aren't that easy to replace.

TimDaub commented 3 years ago

Recommended related reading is Vitalik's post on "The Limits to Blockchain Scalability": https://vitalik.ca/general/2021/05/23/scaling.html

Zer34 commented 3 years ago

We are one of the BSC validator nodes and provide public API services to the BSC community. As we have observed, the volume of transactions on the BSC chain has reached an unprecedented level. We want to share some pointers to help speed up syncing on your full node:

1 Use the latest binary version v1.1.5 and add the --diffsync flag when starting the node 2 Recommended configuration (on AWS): at least c5a.8xlarge with a gp3 disk configured for 12000 IOPS or higher 3 Use the latest official snapshot: https://github.com/binance-chain/bsc-snapshots 4 Configure your node based on the latest config.toml: https://github.com/binance-chain/bsc/releases/download/v1.1.5/mainnet.zip 5 Try temporarily turn off the RPC port to speed up synchronization

If you have more problems with node synchronization, please feel free to utilize our platform (https://infstones.com/) to create a free BSC Public API project or create your dedicated node with only one click.

Bro, are you kidding? Nothing will work on c5a.8xlarge since only the cloud disk is supported there. You won't be able to synch a node there even from a snapshot that appeared an hour ago. And 12k or 16k iops won't help you in any way (I checked it personally myself 2 days ago) . Could you just write us the exact technical parameters of the server that is used for your validator node?

InsideTheSim commented 3 years ago

Maybe time for BSC to become a volition and settle on Ethereum. They they won't have to worry about L1 security — and the inevitable (complete) centralization of BSC will be less of an issue because there will always be an escape hatch to a true L1.

sambacha commented 3 years ago

Hi, this is Stanley at Ankr, which is a web3 infrastructure company hosting one of binance validator and also providing both RPC and staking service to the public. The following are tricks we utilize at Ankr.

  1. We are majorly using bare metals to achieve the best performance given the same level of hardware among major cloud providers.
  2. CPU: AMD Ryzen 5950x is good enough to power both full and archive mode
  3. Memory: DDR4 ECC 128 GB for full, 256 GB for archive. No matter you are using commerical or comsumer based hardware, please make sure using a ECC type of memory!!!
  4. Storage: This is a tricky one. Local NVME would dramatically improve the performance of BSC. So far archive mode is requiring 16TB data and therefore, please leave 30TB for the foreseeable expansion.
  5. Network: This is not critical, a regular 100MB public bandwidth is good enough for syncing from scratch.
  6. Feel free to use any cloud-based VM match the above specs. But according to our experience, bare metals are the ONLY winner for a RPC service provider in terms of cost and performance.

If you are considering a 3rd party RPC service provider, free free to apply one from app.ankr.com. In the meanwhile, Ankr will provide distributed RPC solution for BSC to the public soon. Stay tuned.

To add to @fruit37 comments:

format your ssd's

sudo mkfs.ext4 -F -E stripe-width=256,lazy_itable_init=0,lazy_journal_init=0,discard /dev/md0

Disable write cache flushing

rite cache flushes are fairly slow on local SSDs. You can increase the write performance for some apps by disabling automatic flush commands in those apps or by disabling flush options at the file system level.

To disable write cache flushing on ext4file systems, include the nobarrier setting in your mount options or in your /etc/fstab entries. For example:

sudo mount -o discard,defaults,nobarrier /dev/[LOCAL_SSD_ID] /mnt/disks/[MNT_DIR]

where: [LOCAL_SSD_ID] is the device ID for the local SSD that you want to mount and [MNT_DIR] is the directory in which to mount it.

enable blk-mq

see https://yarondar.wordpress.com/2018/07/29/have-you-tried-blk-mq/

Viable Alternatives - use a different client

There is also this, new alternative client, https://github.com/covalenthq/erigon/tree/chain-bsc-port-latest

Design issues

Also to hijack @TimDaub comment, see this especially pertinent post by Vitalik: toward-a-12-second-block-time

Bare Metal / Clouds

TLDR: use OVH/Equinix/Hive/etc

AWS Inter-Region Latency Map

GCloud Inter-Region Latency Map

Network Latency Performance

To get maximal throughput it is critical to use optimal TCP send and receive socket buffer sizes for the link you are using. If the buffers are too small, the TCP congestion window will never fully open up. If the receiver buffers are too large, TCP flow control breaks and the sender can overrun the receiver, which will cause the TCP window to shut down. This is likely to happen if the sending host is faster than the receiving host (re: this can cause consensus issues or exacerbate existing issues, e.g. out of turn blocks for Clique POA). (lets ignore for a moment why TCP isn't even what you should ideally be using for now)

This is by no means comprehensive but I think it touches on several points that have been outlined here.

ghost commented 3 years ago

Hi, this is Stanley at Ankr, which is a web3 infrastructure company hosting one of binance validator and also providing both RPC and staking service to the public. The following are tricks we utilize at Ankr.

  1. We are majorly using bare metals to achieve the best performance given the same level of hardware among major cloud providers.
  2. CPU: AMD Ryzen 5950x is good enough to power both full and archive mode
  3. Memory: DDR4 ECC 128 GB for full, 256 GB for archive. No matter you are using commerical or comsumer based hardware, please make sure using a ECC type of memory!!!
  4. Storage: This is a tricky one. Local NVME would dramatically improve the performance of BSC. So far archive mode is requiring 16TB data and therefore, please leave 30TB for the foreseeable expansion.
  5. Network: This is not critical, a regular 100MB public bandwidth is good enough for syncing from scratch.
  6. Feel free to use any cloud-based VM match the above specs. But according to our experience, bare metals are the ONLY winner for a RPC service provider in terms of cost and performance.

If you are considering a 3rd party RPC service provider, free free to apply one from https://app.ankr.com. In the meanwhile, Ankr will provide distributed RPC solution for BSC to the public soon. Stay tuned.

Just tried this service for fun. the same delay of 15-30 seconds.

zhongfu commented 3 years ago

Just tried this service for fun. the same delay of 15-30 seconds.

@edgeofthegame just curious, how are you quantifying this delay?

imharvol commented 3 years ago

Just tried this service for fun. the same delay of 15-30 seconds.

@edgeofthegame just curious, how are you quantifying this delay?

If you create an API in Ankr it says this, looks like it desyncs a bit from time to time: image

iduuck commented 3 years ago

It's crazy, how they are still not responding officially to this issue. No "Member" or "Contributor" in this discussion right now.

bostarch commented 3 years ago

Hi, this is Stanley at Ankr, which is a web3 infrastructure company hosting one of binance validator and also providing both RPC and staking service to the public. The following are tricks we utilize at Ankr.

  1. We are majorly using bare metals to achieve the best performance given the same level of hardware among major cloud providers.
  2. CPU: AMD Ryzen 5950x is good enough to power both full and archive mode
  3. Memory: DDR4 ECC 128 GB for full, 256 GB for archive. No matter you are using commerical or comsumer based hardware, please make sure using a ECC type of memory!!!
  4. Storage: This is a tricky one. Local NVME would dramatically improve the performance of BSC. So far archive mode is requiring 16TB data and therefore, please leave 30TB for the foreseeable expansion.
  5. Network: This is not critical, a regular 100MB public bandwidth is good enough for syncing from scratch.
  6. Feel free to use any cloud-based VM match the above specs. But according to our experience, bare metals are the ONLY winner for a RPC service provider in terms of cost and performance.

If you are considering a 3rd party RPC service provider, free free to apply one from https://app.ankr.com. In the meanwhile, Ankr will provide distributed RPC solution for BSC to the public soon. Stay tuned.

128GB? I currently have 64GB bare metal system and regardless what I set, the node will never use more than 32.4GB of ram (commited max is~40GB). How can you effectively use 100+ GB? I'd like to know and use the full hardware.

ghost commented 3 years ago

@zhongfu last accessible block timestamp minus current time (checking a few times per sec). usually it was 1-2 seconds. now it's 10-20 seconds. @bostarch don't bother they have the same lags as everybody else, just tested

ghost commented 2 years ago

Wow guys, reading all these comments really hurts. Why does nobody care about the chain, the network and the people who are running the nodes?!?!!?!? Should be one of the main priorities.

It's also insane to read how immense the hardware requirements are for running a node. It's worse than a SOL node :laughing:

If you are looking for a top team that is building a blockchain with upgradeable smart contracts, a microservice based node, fee less transactions, support for multi languages and many other innovative things... Read about Koinos. Try to set up a node for test net. The only thing you need to do is to start a docker-compose script! Check https://github.com/koinos/koinos or koinos.io

KorNatten commented 2 years ago

I'm not a coder nor do I run a node, I've just been doing my research on where to set up my project for the working class and this is a prime example of what's wrong in the world. Here's a small part quoted from one of my posts I made and why we should change those things mentioned in this repo

I have absolutely nothing against people with a billion $. It’s how many of them, got it and how they use it, that the problem is. Their power and greed gained on the back of the working class people, being used and thrown away as garbage, while governments (put in place by the wealthiest) hope that they die as soon as they get their first pensioncheque. It’s time for wealth sharing according to the capability of a person to do something and what that’s worth to others. What is the value to society of the builders, the person that cleans operating rooms with an eye for detail, factoryworkers making the tools, a surgeon, a media spokesperson of a hospital? Who needs who and do we need that politician who decides that a hospital can be build in that city? Or can people vote on that decision that there should be a hospital there? I doubt anyone in the world has a problem with people with talent, knowledge and skills to be paid accordingly but is that the reality?

Use your time and talents for the people who care! The whole post is here if you agree.

I already had my doubts about the intentions of Binance now I'm really sure I won't be building on their chain either. They just want to be a bank who doesn't care about their workers and customers, like so many others in the current system.

rudy-infstones commented 2 years ago

Bro, are you kidding? Nothing will work on c5a.8xlarge since only the cloud disk is supported there. You won't be able to synch a node there even from a snapshot that appeared an hour ago. And 12k or 16k iops won't help you in any way (I checked it personally myself 2 days ago) . Could you just write us the exact technical parameters of the server that is used for your validator node?

Hey @Zer34, not sure why it didn't work in your case. I just launched another node using the exact specs mentioned; the node was synced and ran stably. Would you mind letting me know if you tried all the listed items at once (latest binary, 'diffsync', latest config, RPC port off, etc.)? I found it crucial to do all on the node for it to work. If you still have synchronization issues, I suggest using an i3.4xlarge or a better instance to get synced faster.

unclezoro commented 2 years ago

I am one of those who commit many Pull Requests in the BSC repo. We are keeping our heads down on building and optimizing the BSC to meet the requirements from 2M daily active addresses (ATH recently)

I apologize that we did not put more effort to improve enough transparency and communication for each release.

However, we do care a lot for the value of the community, and really appreciated your critizations and suggestions. We will work with our supportive community to keep improving the stability of infrastructure and introduce new scalable solutions.

Let me have some summary here:

  1. BSC node sync issue: Some validators like Ankr and InfStones gave more detailed setups. Here are a few proposed tips - https://github.com/binance-chain/bsc/issues/338; https://github.com/binance-chain/bsc/issues/502. State prune is one key action that many have missed and didn’t perform as a routine. After rigorous testing, BSC has released a Bruno Upgrade 1.1.5. With Diff Sync, the node sync speed has improved substantially as we have witnessed in many nodes. More architecture related improvements are under their way to be proposed as new BEPs soon. Please check it out and let us know your thoughts! If you still have issues, please join node operators Telegram https://t.me/joinchat/n2s046C1_OMxMTU1 or discord: https://discord.gg/J4HUc9zK . You will get more support from the hearty BSC community.
  2. BSC is open-sourced since its day one and all of the BSC codes are transparent and viewable for reviewing. However, the BEP Process should be emphasized to improve the transparency of each and every change/update we plan to deploy and improve our overall communication. If you have any specific proposal, please do propose. BSC is currently one of the most active blockchains with huge amounts of active addresses and largest transactions after several rounds of performance enhancement. This has resulted in new challenges and we require more support for better research, development and engineering. We are really looking forward to seeing more strong developers here to join our development together. This is a great example from the community: https://github.com/ledgerwatch/erigon/pull/2991. Please propose more suggestions to help us improve the infra.

We'll keep utilizing Github as a builder collaboration platform to raise the proposal, create concrete issues with facts (e.g., logs), discuss solutions, and build infra. For other types of discussion, join our public communities and feel free to ping our admins whenever necessary.

Node discussion discord: https://discord.gg/gx9E5fCvmW Dev discussion discord: https://discord.gg/E6yDKuV5MT

Thank you for your patience and support!

quinnj102 commented 2 years ago

Come on Build on Bitcoin. The future is there!

why would anyone want to build on bitcoin with 10 min block times?

Relicy commented 2 years ago

I've switched to Solana

hellodword commented 2 years ago

Bro, are you kidding? Nothing will work on c5a.8xlarge since only the cloud disk is supported there. You won't be able to synch a node there even from a snapshot that appeared an hour ago. And 12k or 16k iops won't help you in any way (I checked it personally myself 2 days ago) . Could you just write us the exact technical parameters of the server that is used for your validator node?

Hey @Zer34, not sure why it didn't work in your case. I just launched another node using the exact specs mentioned; the node was synced and ran stably. Would you mind letting me know if you tried all the listed items at once (latest binary, 'diffsync', latest config, RPC port off, etc.)? I found it crucial to do all on the node for it to work. If you still have synchronization issues, I suggest using an i3.4xlarge or a better instance to get synced faster.

@rudy-infstones Can you give your instance type, system configurations, config.toml, all commands you execute till the node is syncing? I want to follow your steps strictly. Thanks.

ratthakorn2509 commented 2 years ago

Binance Smart Chain Binance Chain I would like to know how much of my assets I have now and how much farming investment I have. I know the details, what should I do? Thank you very much.

rudy-infstones commented 2 years ago

We are one of the BSC validator nodes and provide public API services to the BSC community. As we have observed, the volume of transactions on the BSC chain has reached an unprecedented level. We want to share some pointers to help speed up syncing on your full node:

1 Use the latest binary version v1.1.5 and add the --diffsync flag when starting the node 2 Recommended configuration (on AWS): at least c5a.8xlarge with a gp3 disk configured for 12000 IOPS or higher 3 Use the latest official snapshot: https://github.com/binance-chain/bsc-snapshots 4 Configure your node based on the latest config.toml: https://github.com/binance-chain/bsc/releases/download/v1.1.5/mainnet.zip 5 Try temporarily turn off the RPC port to speed up synchronization

If you have more problems with node synchronization, please feel free to utilize our platform (https://infstones.com/) to create a free BSC Public API project or create your dedicated node with only one click.

Bro, are you kidding? Nothing will work on c5a.8xlarge since only the cloud disk is supported there. You won't be able to synch a node there even from a snapshot that appeared an hour ago. And 12k or 16k iops won't help you in any way (I checked it personally myself 2 days ago) . Could you just write us the exact technical parameters of the server that is used for your validator node?

Hey @Zer34, not sure why it didn't work in your case. I just launched another node using the exact specs mentioned; the node was synced and ran stably. Would you mind letting me know if you tried all the listed items at once (latest binary, 'diffsync', latest config, RPC port off, etc.)? I found it crucial to do all on the node for it to work. If you still have synchronization issues, I suggest using an i3.4xlarge or a better instance to get synced faster.

@rudy-infstones Can you give your instance type, system configurations, config.toml, all commands you execute till the node is syncing? I want to follow your steps strictly. Thanks.

Hey @hellodword , please reference my previous comment regarding the setup. We used c5a.8xlarge and the exact official config.toml. Feel free to let me know if you have any further questions. You can also try to set up a BSC node on our platform (https://cloud.infstones.io/)

perfectcircle2020 commented 2 years ago

guys despite all the problems the network has currently it reached huge milestones, the fact that people only complain and dont try to come up with practical and implementable solutions makes me sad, even if you are not a coder or programmer, try at least to read and understand the basics of blockchain technology and put in some effort to think what might be the problem for your current situation and talk publicly providing insights and useful information that other people can learn from, only complaining wont speed up the sync and make the network better thats for sure.

let this moment be a uniting moment for all of us, when a new phase of bsc begins.

ghost commented 2 years ago

here for the memez. 5v5uff 5v5uur 5v5v8g

its5Q commented 2 years ago

@edgeofthegame jokes aside, once my server's disk died from running 2 nodes at the same time and the hosting provider had to replace it

0xyolo commented 2 years ago

At C.R.E.A.M. we’ve been running our BSC validator node on AWS EC2 m5zn.3xlarge with 2.5 TB gp3 SSD 3000 IOPS 125 MB/s and it’s been very stable for a long time. We started seeing sync issues starting November 3rd and started missing blocks then. Last week we enabled the new diff sync feature from BSC and upgraded the disk performance to 8000 IOPS, 250MB/s and our node has been stable ever since. We recommend others to try the same. Like any cutting edge technology, there are often challenges in keeping up with the latest and greatest. While this thread and discussion has spawned many valid concerns, we believe this is a healthy discussion around how BSC can improve moving forward. We agree with the need for a shared channel where validators can learn from one another and we’re glad to see BSC has already deployed this. Looking forward to more innovations to come!

miohtama commented 2 years ago

BSC, and its issues, stems from GoEthereum that is 2014 tech. BSC can improve, but only if they invest in building a more open and healthy community. For the sync issues, they are very real and I wrote a Twitter thread here:

https://twitter.com/moo9000/status/1463454127095042050

(Contains some useful tips for those who are struggling with syncing.)

CryptoManiac commented 2 years ago

Just done syncing a new node on Micron SATA drive, it took less than 14 hours. So there is no need for NVMe or whatever. SATA bandwidth is more than enough (20x more than needed actually) and latency is not much of an issue as it seemed to be.

zhongfu commented 2 years ago

@CryptoManiac no need, yes, but it's really really not recommended -- you'd very likely see it falling behind quite often

what's the mgasps like on your node anyway

CryptoManiac commented 2 years ago

@CryptoManiac no need, yes, but it's really really not recommended -- you'd very likely see it falling behind quite often

Well I can say that NVMe wouldn't be of much help anyway. I spent a week testing and comparing various hardware and came to this exact realization that enterprise-grade SATA drive is more than enough.

what's the mgasps like on your node anyway

I sometimes see 500-700 mgasps but it's usually between 60 and 200. It's like a dice. :)

CryptoManiac commented 2 years ago

Ubuntu Linux 20.04 on 8-core Xeon and 32 Gb RAM. I tried to install 64 Gb and check the results and can say that it wouldn't hurt but rather pointless.

No tweaks or whatever, everything is literally running at default settings including the BSC client itself. Perhaps the main difference from other reports is that it's not a virtual machine but a physical hardware.

IMG_0650

P.S. I forgot to mention that system and the BSC data folder are located on Micron 5300 Pro 3.84Tb MTFDDAK3T8TDS.