search

near / nearcore

Reference client for NEAR Protocol
https://near.org
GNU General Public License v3.0
2.33k stars 626 forks source link

Consider using higher-performance SSD. Launch new IO fees #4461

Closed MaksymZavershynskyi closed 2 years ago

MaksymZavershynskyi commented 3 years ago

We have re-calculated IO costs and we discovered that they are more expensive than we thought before.

We should consider using higher-performance SSD to match the state of art on other blockchains.

Regardless of whether we switch our hardware requirement to higher-performance SSD or not we should update our protocol to use the new fees.

CC @janewang

vans163 commented 3 years ago

NVMEs cost as much as SSDs these days (unless you need really high density 4TB+) and have almost 10x more IOPS.

The costs should also not be based off AWS / Public Clouds, but off running normal dedicated servers / computers and regular IP transit (IXPs for example but this is too low a cost).

The cost for someone who has a 1gbps line to run a small node (like 4-8 core proc, 16G ram, 2tb NVME) is much less than running the same thing on AWS, mostly due to the 1000x markup bandwith costs clouds charge (compared to bandwith prices at a IXP).

bowenwang1996 commented 3 years ago

I believe we use "persistent SSD" on gcloud for benchmarks. They are likely much slower than local SSDs. We should potentially try that.

vans163 commented 3 years ago

I believe we use "persistent SSD" on gcloud for benchmarks. They are likely much slower than local SSDs. We should potentially try that.

Can these benchmarks be done on metal OVH/Hetzner/Packet servers instead of virtualized cloud servers that have degraded unmeasurable performance? (impossible to predict how over provisioned the metal running the virtual server is during the benchmark)

bowenwang1996 commented 3 years ago

Can these benchmarks be done on metal OVH/Hetzner/Packet servers instead of virtualized cloud servers that have degraded unmeasurable performance? (impossible to predict how over provisioned the metal running the virtual server is during the benchmark)

I believe quite a few validators still use cloud providers to run nodes. We'd like to make sure that would be still feasible

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

matklad commented 2 years ago

@jakmeier I think the status quo changed significantly since we created this issue. Do you think we should close it?

jakmeier commented 2 years ago

I think closing is the right decision here.

But for context, let me mention the two big efforts that are currently ongoing to address the underlying concern here.

  1. Flat storage is supposed to reduce IO costs significantly AND make it much more predictable. It might also lead to a change in what parameters we use to charge IO costs. Therefore, we postpone fixing costs until that is implemented. (#7327)
  2. IO costs are currently the highest for archival nodes, which fall behind anytime there is lots of IO load. Cold storage implementation should help with that, as it will probably change what SSDs validators are using. And it changes what we could feasibly asked them to use. Thus, making any decision on current data is probably misguided.

With these two efforts in mind, I think we can close this old issue.