CosmWasm / wasmd

Basic cosmos-sdk app with web assembly smart contracts
Other
367 stars 398 forks source link

Document recommended hardware requirements #877

Closed ethanfrey closed 1 year ago

ethanfrey commented 2 years ago

We only define the chipsets needed (intel/amd64) but not RAM/Disk/etc.

wasmd runs quite well on low power CI machines. But it needs a lot more power for a heavily used mainnet. Also different specs for validators and sentry/archive nodes.

I would like to ask chains running CosmWasm in production to comment on recommended hardware settings, so we can provide some general advice to other chains that want to add it.

faddat commented 2 years ago

I can put together a guide based on experience with Juno and osmosis.

64 or 128 GB of RAM has always seem like a good path to me. Additionally, having an NVME hard drive matters.

should this go in readme.md?

can also give detail on sentry configs and the like.

ethanfrey commented 2 years ago

64 or 128 GB of RAM has always seem like a good path to me.

We have run the CosmWasm testnets with 2cpu / 4GB RAM machines. Of course those are much lower traffic than Osmosis or Juno. I would like some numbers Re: transactions or usage and the RAM needed.

Those 64/128 GB of RAM for Osmosis is a chain that doesn't run any smart contracts yet. So not really CosmWasm numbers. I really wish there was a benchmark of how much a "pure Go" SDK app needed at certain usage and then we can show what CosmWasm needs at similar usage.

And yes, SSD is very important.

I synced a node on Osmosis with 32GB RAM without problem. It was super slow on HDD, but fine on SSD. Of course, I was not serving a public RPC endpoint. I feel there are so many variables more important than CosmWasm or not. Anyone know of some reference for the normal SDK without CosmWasm?

faddat commented 2 years ago

Hey Ethan, I think that it's just too variable.

For the SDK itself, well first of all I suspect strongly that it leaks RAM. Secondly, I can't find where it leaks the ram, I just know that when I validate nomic, my nodes do not leak RAM. When validating other chains they most certainly do and the biggest difference is the use of the SDK.

I strongly suspect that contract chains are going to behave as you have described, the load will greatly impact the system requirements. If you would like, I can outline what we do at notional, but the trouble is that tends to be highly specific on a per chain basis.

alpe commented 2 years ago

Some examples from chains:

Recommended for the hub:

Node Type RAM Storage
Validator 32GB 500GB-2TB*
Full 16GB 2TB
Default 16GB 1TB

Mininum for Juno:

Minimum for Stargaze

alpe commented 1 year ago

Should we simply drop this issue and consider this the responsibility of the chains running with the wasmd module? There was not much feedback on our discord. The hardware requirements depend on the SDK version, other modules and traffic as well. The minimum on Juno looks reasonable but I was running wasmd testnets with validator nodes that had only 2 CPU and 8GB RAM, too.

alpe commented 1 year ago

As discussed on the call: there are too many variables so we leave this to the chain architects to be defined for their use cases