Scaling: contract-size and storage limits

[dusterbloom]

Summary

This issue came up as we worked on DCAP and the tcbinfo contract. The latter in particular proved difficult to first upload then instantiate due to its inclusion of heavy dependencies with all default-features not turned to false.

It follows that we need to figure out if our current approach of having one contract / one enclave in combination can lead to problems with contract - size whether at instantiation or later on at migrations or upgrade.

Open questions:

1. How big can a contract be?
2. Is the approach one contract - one enclave sustainable in scaling terms?

Acceptance Criteria

• [x] We know the hard limits of cosmwasm
• [x] SOTA contract management strategies in the ecosystem (neutron, stargaze? et al.)
• [x] Define a scaling strategy that avoid hitting the limits for contract size

[dusterbloom]

Overview of cosmwasm limits

• 800KB / 1MB per contract size limit for wasm binaries
• Gas usage: It varies from chain to chain. Complex contracts consume lots of gas, which can be a bottleneck, particularly when using resource-heavy cryptographic libraries.

Strategies to stay within the limits

Contract size

1. Optimize -> Always use Rust Optimizer, opt and all available tools to minimize binary size
2. Pin -> Always pin contracts that are needed frequently as it reduces gas costs quite significantly (we should start listing which contracts we might prefer pinned)
3. Modularize -> Keep each contract as simple as possible while relying on multi contract interaction

Tips from the docs:

1. serde_json_wasm instead of serde_json -> this should reduce the contract size
2. Pin also light clients and/or contract executed at begin / end of block

Pinning: what is it? how to use it?

It ensures that a previously stored compiled contract is available and started from in-memory cache rather than from disk.

HowTo:

1. Rust -> To pin/unpin a contract, use [Cache::Pin] / [Cache::Unpin]
2. wasmd -> implements InitializePinnedCodes
3. gov module -> implements MsgPinCodes / MsgUnpinCode which makes pinning / unpinning work by voting

State Storage:

CosmWasm contracts store their state in the underlying chain’s storage. Large state usage can increase costs or lead to gas constraints during operations like migrations or upgrades.

While there's no universal "hard" limit on state storage size, larger storage leads to higher gas consumption, especially during migrations or upgrades. Hence, state-heavy contracts should be designed to minimize on-chain storage in favour of off-chain storage (IPFS or other).

---

On quartz:

The current approach of one contract <-> one enclave binary <-> one websocket listener is an idealization of a much more complex system which we have actually developed.

In order to deploy a quartz app, the app developer needs:

• a chain in which contracts can be instantiated
• a way to start, init and handshake the enclave
• a front end running for users to interact with it
• a relay is working between the above / a websocket listening for relevant events

Current setup for the transfers app includes:

1. Smart Contracts:

   • Transfers Contract
   • CW4 Group membership
   • TCBInfo Contract

2. Enclave binaries:

   • Transfers Enclave

3. Listener:

   • Websocket listener as part of the cli

4. Core Components:

   • Quartz Common Library
   • Quartz CLI

5. Supporting Components:

   • CosmWasm Integration
   • DCAP Attestation
   • Light Client
   • Relay

Current limitations:

The current setup is limited in that it forces the developer to work on a single contract / binary / listener paradigm. This might lead to creating contracts whose binary size might exceed the current size limit of 800KB / 1MB.

SOTA

Currently on neutron the biggest deployed core contract is around 200KB. All other contracts I saw were rarely larger than than. Link to neutron pinned contract list

Open issues that need more investigation

1. Contract Investigate how to leverage pinned contracts , proxy contracts and factory contracts work and what impacts would they have on our current setup.
2. TEE / DevOps - A multi-enclave scenario -> multiple contracts deployed calling more than one enclave depending on work type -> Is this doable? How hard is orchestrating multiple enclaves? Would this make quartz apps composable somehow?
3. DevX - As a dev I'd love to be able to build something which can offer functionality from more than one contract / app / enclave / websocket listener at any single time
4. (shoutout to @dangush ) DevX -As a dev I'd love to easily customize events to which I want my app to subscribe and also having the option to do deep customization if needed

[dangush]

Nice writeup @dusterbloom , we have a lot of room to think about architecture now that we've gotten a better look at how Quartz is shaping up.

One note I'd like to add is that the paradigm currently is actually more like 1 contract, 1 enclave, plus 1 websocket listener. The last one is referring to what used to be the listen bash scripts, which are now Rust code implemented in wslistener.rs in every quartz app.

This websocket listener logic is relatively low level, as it requires indexing into tendermint events and calling the tm-prover library. It would be best if we provided clean abstractions so that users could implement basic/common websocket listener logic seamlessly, but with room for deep customization. Where we want to sit on the simplicity-flexibility spectrum is an open question.. maybe it will depend on what we see users asking for in public. watchexec seems like a good library to draw inspiration from given the similarity of the problem it solves. They definitely sided with the "flexibility" side of the spectrum, so there isn't too much structure to using their library, but their good documentation makes it easy enough to use.

Also, quartz common just re-exports quartz-{contract, enclave, proto} so technically from the user perspective, its the only lib they have to install :)

[dusterbloom]

@dangush really like the idea of giving devs clean abstractions so that they could implement basic/common websocket listener logic seamlessly, but with room for deep customization. Didn't know about watchexec will check it out.