Rework deployment infrastructure

fredo commented 2 years ago

We need to make it easy to deploy Beamer contracts on new chains, or redeploy on chains where a deployment already exists. To that end, a split approach like the following may prove useful.

New deployment directory structure

The new deployments directory structure would look like this:

deployments/                         
├── config                           
│   └── mainnet
│       └── rpc.json
│       └── 288-boba.json                
│       └── 10-optimism.json            
│   └── goerli
│       └── rpc.json
│       └── 2888-boba.json                
│       └── 599-metis.json               
├── artifacts                           
│   └── mainnet                      
│       └── base.deployment.json   
│       └── 288-boba.deployment.json     
│       └── 10-optimism.deployment.json 
│   └── goerli                      
│       └── base.deployment.json   
│       └── 2888-boba.deployment.json

The config dir is where all the deployment config files are stored, organized per layer1 chain. The artifacts dir is where all the deployment artifacts are stored, again, organized per layer1 chain.

Note the absence of contract ABI files. The ABI files will be obtained by another script that reads deployment file(s) and outputs ABI files. For that reason, it is important that each .deployment.json file has a beamer_commit entry, e.g.

beamer_commit": "3fc75ed6e281309d9c504d9794874a7f6500a5fe",

The deployment process

The overall deployment process can be split into 3 distinct actions, each available as a separate `beamer` command: *	`beamer <command>`	Description	Inputs	Outputs	Affected chains
1	`deploy-base`	deploy Resolver	`rpc.json`	`base.deployment.json`	L1
2	`deploy`	deploy chain-specific contracts and set up trusted call chain	`rpc.json`, `base.deployment.json`	`$chain.deployment.json`	L1, L2
3	`configure`	configure contracts	all configuration files, all deployment artifacts		L2

Contract-related beamer commands

In command examples that follow, options --keystore-file, --password and --rpc-file have been omitted for brevity.

Note: in sections below, CHAIN refers to a concatenation of a chain ID and a human-readable name, {chain-id}-{name}. For example, 288-boba or 10-optimism.

`beamer deploy-base --artifacts-dir ARTIFACTS_DIR CHAIN_ID`

Deploy the Resolver contract on the base chain. The deployment information is written to $ARTIFACTS_DIR/base.deployment.json. Example:

beamer deploy-base --artifacts-dir deployments/artifacts/mainnet 1

`beamer deploy --artifacts-dir ARTIFACTS_DIR CHAIN.json [CHAIN.json]...`

Deploy L2 Beamer contracts on the specified chains and set up the trusted call chain.

The working of this command can be roughly described as follows. Preparation:

read $ARTIFACTS_DIR/base.deployment.json to get Resolver deployment info

For each given configuration file:

deploy all needed contracts on chain and write the deployment info to $ARTIFACTS_DIR/$CHAIN.deployment.json
deploy the chain-specific L1 messenger contract on the base chain and add the deployment info to $ARTIFACTS_DIR/base.deployment.json
set up the trusted call chain

Example:

beamer deploy --artifacts-dir deployments/artifacts/mainnet
       deployments/config/mainnet/288-boba.json
       deployments/config/mainnet/10-optimism.json

The above example will produce:

  deployments/artifacts/mainnet/288-boba.deployment.json
  deployments/artifacts/mainnet/10-optimism.deployment.json

`beamer configure --config-dir CONFDIR CHAIN.deployment.json [CHAIN.deployment.json]...`

Configure contract data specified by the config file(s). The working of this command can be roughly described as follows.

For each given deployment file CHAIN.deployment.json:

for each X.json in CONFDIR:
- if X == CHAIN, do nothing;
- otherwise:
- read configuration from $CONFDIR/X.json
- update appropriate contract data of X in CHAIN

Example:

beamer configure --config-dir deployments/config/mainnet
       deployments/artifacts/mainnet/288-boba.deployment.json
       deployments/artifacts/mainnet/10-optimism.deployment.json

taleldayekh commented 2 years ago

@istankovic, can you add a comment on what needs to be done so that we can estimate this in the refinement?

istankovic commented 2 years ago

We need to make it easy to deploy Beamer contracts on new rollups, or redeploy on on rollups where a deployment already exists. To that end, a split approach like the following may prove useful.

Split the deployment into the several phases:

1) deploy Resolver on L1 2) for each L2, deploy it and construct the trusted call chain (this is possible since L1 resolver is deployed by now) 3) initialize the finality periods for all chains

The new deployments directory structure would look like this:

deployments/                         
├── config                           
│   └── mainnet                      
│       └── layer1.json              
│       └── boba.json                
│       └── optimism.json            
│   └── rinkeby                      
│       └── layer1.json              
│       └── boba.json                
│       └── metis.json               
├── output                           
│   └── mainnet                      
│       └── layer1.deployment.json   
│       └── boba.deployment.json     
│       └── optimism.deployment.json 
│   └── rinkeby                      
│       └── layer1.deployment.json   
│       └── boba.deployment.json

The config dir is where all the deployment config files are stored, organized per layer1 chain. The output dir is where all the deployment output files are stored, again, organized per layer1 chain.

Note the absence of contract ABI files.

:warning: :construction: :construction_worker: :warning:

GabrielBuragev commented 2 years ago

I think having the file names as {chainId}.deployment.json instead of {chainName}.deployment.json would make this approach more consistent and more integrable with the other components in the repo (frontend for example).

fredo commented 2 years ago

I think having the file names as {chainId}.deployment.json instead of {chainName}.deployment.json would make this approach more consistent and more integrable with the other components in the repo (frontend for example).

100% agree. I already mentioned that to Ivan. I think it is even a necessity because we get the chain id from the RPC and act based on that. At the points where it is needed we can to a "human readable translation" though

istankovic commented 2 years ago

I think having the file names as {chainId}.deployment.json instead of {chainName}.deployment.json would make this approach more consistent and more integrable with the other components in the repo (frontend for example).

100% agree. I already mentioned that to Ivan. I think it is even a necessity because we get the chain id from the RPC and act based on that. At the points where it is needed we can to a "human readable translation" though

I guess the main question here revolves around whose job we would like to make easier -- the computer's or the developer's? While I do not have a too strong opinion here, I do think that using human-readable names makes it easier for us

to manage the deployments
quickly navigate our way through the deployment directories

In some sense, the current structure could also be seen as biased toward being easier for humans to handle. Note that nothing stops us from storing chain IDs in the deployment files -- that was actually what I was planning to propose, just did not have the time to finish this issue. It would also mean one would need to read the entire L1-specific directory in order to find deployment file with a given chain ID, but that will likely need to be done anyway in most cases, is not on the performance-sensitive path and would make the computer do more work, not the human.

At the points where it is needed we can to a "human readable translation" though

You probably mean to do this translation during runtime, while I'm completely ignoring runtime here and instead am focusing on things a human might need when debugging, preparing to (re)deploy Beamer on all or a subset of rollups etc. For example, if someone asks "which rollups has Beamer been deployed to on $L1?", it is much nicer to respond with rollup names directly, rather than have to do the chain ID lookup or, worse, just respond with numbers.

fredo commented 2 years ago

Actually I'm not sure if we are doing us or anybody else a favor. Let me elaborate: I understand that the agent is doing a non performance critical execution while parsing the files. I'm not really concerned about the performance. What we are trying to do when creating human readable directory names is finding a human readable identifier for a certain rollup. But in fact, this doesnt exist per convention in the ecosystem, and we are the ones maintaining it. While we have now a clear and distinct set of rollups in the future names might be very similar. As an example, Arbitirum is launching multiple products right now. There is Arbitrum nitro , Arbitrum Nova and what not. I know that other roll ups are planning the same. Luckily there is also a canonical identifier at least per convention in the Ethereum Ecosystem. This also accounts for humans. Namely the chain id. We can be somewhat sure that there will never be another network with the same chain id (Otherwise we would have a completely but severe problem). Imagine that we are naming the directory Arbitrum, now one could ask, are you talking about nitro, nova or version xyz? At the end the user, would simply look at the chain id and get the answer. Of course, it is our job to keep the names up to date and distinguishable and the whole purpose of naming the the directory is that a human can look at the directory name and knows exactly what we are talking about. So if names are not 100% distinguishable then the purpose failed. On the other hand it is best practise and well known in the space that unique identifiers are chain ids, so for example, I look more on chain ids than on actual names in my daily work.

Nonetheless, I totally get your point, but I think not having the chain id in the directory name will not fulfill your requirement in the future. But as a compromise, what speaks against a combination of the two? Something like {chainId}-{chainName}.deployment.json? I would be also totally happy with something like this.

istankovic commented 2 years ago

[...]

Some very good points, thanks for elaborating and helping me understand.

Something like {chainId}-{chainName}.deployment.json? I would be also totally happy with something like this.

Totally fine with that. :+1:

(Just to illustrate my struggle: I learnt that 10 is Optimism, but given 28 and 588 I always have to do a look up to see which one is Metis and which one is Boba. WIth more rollups, that will only get worse. :sweat_smile: )

fredo commented 2 years ago

I totally understand the struggle. I think we will have a good compromise with the approach. Let's got with that

istankovic commented 2 years ago

I have created individual stories and fleshed out the design in a bit more detail, in particular in #972.

fredo commented 2 years ago

for consistency, I would call layer1.json , 1-ethereum.json.

beamer-bridge / beamer