search

commerceblock / mercurylayer

A Blinded implementation of mercury statechains
53 stars 9 forks source link

Multiple enclave support #59

Closed tomt1664 closed 1 week ago

tomt1664 commented 2 weeks ago

For future scaling:

The mercury server is stateless (state in DB) and so can be easily replicated in deployment to handling increases in demand. The enclave cannot be replicated - single bare metal instance provisioned in advance.

However it is possible to scale the number of enclaves deployed manually - each enclave only needs to be unique to a specific statechain_id. Different shared keys (statechain_ids) can be handled by different enclaves.

To implement this, the server can configured with more than 1 enclave (i.e. and array of enclave URLs). Each shared key (i.e. statechain_id) when it is first created is assigned to one of the enclaves (based on some user data or random - e.g. ordering derived from the initial auth_pubkey). The enclave index (i.e. which URL in the array) is then recoded in the DB in an additional column. When functions for that statechain_id are called in the future, the correct enclave URL to use is found from the array and the index in the DB.

This should work as the server config is updated. E.g. initially a single enclave is specified. All statechain_ids use this single enclave (index 0). Later on, a second enclave is added to the server config and restarted. Now there are two enclave URLs (index 0 and index 1). New keygens/deposits are assigned randomly either index 0 or index 1. Eventually (as people withdraw coins) load should be spread evenly across the two enclaves.

This was implemented in the old mercury server: https://github.com/commerceblock/mercury/blob/e5ef3cd8cdb8a5cf32696d646d602953e5f4852a/server/src/protocol/util.rs#L551

tomt1664 commented 1 week ago

Another feature which would be potentially useful is a way to remove or 'retire' an enclave. E.g. if an enclave fails (and any backup also fails) new new keygens/deposits should be forwarded to that enclave. Also, e.g. we want to migrate over a longer period, so we want to keep a specific enclave operating (for existing coins) but new all deposits should not utilise the specific enclave (so after the lifetime of a coin, that enclave can be safely shut down).

ssantos21 commented 1 week ago

Implemented in #61