search

nomad-xyz / nomad-monorepo

Contracts, off-chain agents, and libraries for Nomad
https://nomad.xyz
77 stars 16 forks source link

design: unify configuration files into single source of truth #69

Open yourbuddyconner opened 2 years ago

yourbuddyconner commented 2 years ago

We currently have several sources of configuration for the various parts of the stack, it makes for a really annoying time when performing housekeeping tasks like updating contracts after a deployment.

Currently we have a web of dependencies starting with the rust configs and contract metadata outputted by nomad-deploy. Tools and applications that get configured based on the output of nomad-deploy are:

Proposal:

Build a single config file that acts as the single source of truth for the entire stack. It should contain all necessary data about the shape of a deployment, including fields for:

These config files (one for each environment) would be persisted in the monorepo, referenced by the SDK, and any downstream packages that use the SDK will be automagically configured upon SDK updates.

anna-carroll commented 2 years ago

default agent parameters (?) default agent signers (?)

wdym by this?

anna-carroll commented 2 years ago

sketched a flow of what it would look like to automate this process IMG_5526

anna-carroll commented 2 years ago

related: https://github.com/nomad-xyz/nomad-monorepo/issues/42

yourbuddyconner commented 2 years ago

default agent parameters (?) default agent signers (?)

wdym by this?

Agent configs currently have placeholders for signer keys, want to ensure this new thing has parity there.

yourbuddyconner commented 2 years ago

Took a crack at unifying this today, here's an example of that the mainnet config could look like: 

{
    "environment": "production",
    "networks": ["ethereum", "moonbeam"],
    "rpcs": {
        "moonbeam": ["https://moonriver.api.onfinality.io/public"],
        "ethereum": ["https://main-light.eth.linkpool.io/"]
    },
    "core": {
        "ethereum": {
            "name": "ethereum",
            "domain": 6648936,
            "connections": ["moonbeam"],
            "consensus": {
                "blockTime": 15,
                "optimisticSeconds": 1800
            },
            "contracts": {
                "upgradeBeaconController": "0xdB378579c2Af11817EEA21474A39F95B5b9DfD7e",
                "xAppConnectionManager": "0xFe8874778f946Ac2990A29eba3CFd50760593B2F",
                "updaterManager": "0x9272C9d5fa902Ef3804EC81e0333Ae420D57f715",
                "governanceRouter": {
                    "implementation": "0x569D80f7FC17316B4C83f072b92EF37B72819DE0",
                    "proxy": "0x3009C99D370B780304D2098196f1EBF779a4777a",
                    "beacon": "0x67833a48b3F509d4252ac2c19cd604556eD6c981"
                },
                "home": {
                    "implementation": "0x8F184D6Aa1977fd2F9d9024317D0ea5cF5815b6f",
                    "proxy": "0x92d3404a7E6c91455BbD81475Cd9fAd96ACFF4c8",
                    "beacon": "0x063e871f8DB991CEAd34B557A00B157B360084cc"
                },
                "remoteReplicas": {
                    "moonbeam": {
                        "implementation": "0x67833a48b3F509d4252ac2c19cd604556eD6c981",
                        "proxy": "0x7F58bb8311DB968AB110889F2Dfa04ab7E8E831B",
                        "beacon": "0x3009C99D370B780304D2098196f1EBF779a4777a"
                    }
                }
            },
            "governance": {
                "governor": {
                    "address": "0x93277b8f5939975b9e6694d5fd2837143afbf68a",
                    "domain": 6648936
                },
                "recoveryManager": "0xda2f881f7f4e9d2b9559f97c7670472a85c1986a",
                "recoveryTimelock": 86400
            },
            "updaters": ["0x71dC76C07E92325e7Cc09117AB94310Da63Fc2b9"],
            "watchers": ["0x9782A3C8128f5D1BD3C9655d03181ba5b420883E"],
            "agents": {
                "rpcStyle": "ethereum",
                "timelag": 20,
                "db": "/path/to/db",
                "logging": {
                    "level": "debug",
                    "fmt": "json"
                },
                "index": {
                    "from": 13983724,
                    "chunk": 2000
                },
                "updater": {
                    "enabled": true,
                    "signers": {
                        "transaction": {},
                        "attestation": ""
                    },
                    "interval": 5
                },
                "relayer": {
                    "enabled": true,
                    "signers": {
                        "transaction": {}
                    },
                    "interval": 10
                },
                "processor": {
                    "enabled": true,
                    "signers": {
                        "transaction": {}
                    },
                    "interval": 5
                },
                "watcher": {
                    "enabled": true,
                    "signers": {
                        "transaction": {},
                        "attestation": ""
                    },
                    "interval": 10
                },
                "kathy": {
                    "enabled": false,
                    "signers": {},
                    "interval": 100
                }
            }
        },
        "moonbeam": {
            "name": "moonbeam",
            "domain": 1650811245,
            "connections": ["ethereum"],
            "consensus": {
                "blockTime": 15,
                "optimisticSeconds": 1800
            },
            "contracts": {
                "upgradeBeaconController": "0xdCe06fFE78AaAc2894109A56BA83C3C33B073F44",
                "xAppConnectionManager": "0xdB378579c2Af11817EEA21474A39F95B5b9DfD7e",
                "updaterManager": "0x2e09EdD238EeaEA6e4da705fbe5922B1979e03aC",
                "governanceRouter": {
                    "implementation": "0x7B39dA90C9eAF87e85C553964BC3CBd674e7CCc1",
                    "proxy": "0x569D80f7FC17316B4C83f072b92EF37B72819DE0",
                    "beacon": "0xA84e233A12b36125A731e1362121d8d4ea030c91"
                },
                "home": {
                    "implementation": "0x9272C9d5fa902Ef3804EC81e0333Ae420D57f715",
                    "proxy": "0x8F184D6Aa1977fd2F9d9024317D0ea5cF5815b6f",
                    "beacon": "0xFe8874778f946Ac2990A29eba3CFd50760593B2F"
                },
                "remoteReplicas": {
                    "ethereum": {
                        "implementation": "0x7F58bb8311DB968AB110889F2Dfa04ab7E8E831B",
                        "proxy": "0x049b51e531Fd8f90da6d92EA83dC4125002F20EF",
                        "beacon": "0x0876dFe4AcAe0e1c0a43302716483f5752298b71"
                    }
                }
            },
            "governance": {
                "governor": {
                    "address": "0x93277b8f5939975b9e6694d5fd2837143afbf68a",
                    "domain": 6648936
                },
                "recoveryManager": "0xea24Ac04DEFb338CA8595C3750E20166F3b4998A",
                "recoveryTimelock": 86400
            },
            "updaters": ["0x40FD91557B318BD5d52D12535795265c88702681"],
            "watchers": ["0x297BBC2F2EAAEB17Ee53F514020bC8173F0570dC"],
            "agents": {
                "rpcStyle": "ethereum",
                "timelag": 20,
                "db": "/path/to/db",
                "logging": {
                    "level": "debug",
                    "fmt": "json"
                },
                "index": {
                    "from": "171256",
                    "chunk": "2000"
                },
                "updater": {
                    "enabled": true,
                    "signers": {
                        "transaction": {},
                        "attestation": ""
                    },
                    "interval": 5
                },
                "relayer": {
                    "enabled": true,
                    "signers": {
                        "transaction": {}
                    },
                    "interval": 10
                },
                "processor": {
                    "enabled": true,
                    "signers": {
                        "transaction": {}
                    },
                    "interval": 5
                },
                "watcher": {
                    "enabled": true,
                    "signers": {
                        "transaction": {},
                        "attestation": ""
                    },
                    "interval": 10
                },
                "kathy": {
                    "enabled": false,
                    "signers": {},
                    "interval": 100
                }
            }
        }
    },
    "bridge": {
        "ethereum": {
            "contracts": {
                "bridgeRouter": {
                    "implementation": "0xD3dfD3eDe74E0DCEBC1AA685e151332857efCe2d",
                    "proxy": "0x88A69B4E698A4B090DF6CF5Bd7B2D47325Ad30A3",
                    "beacon": "0xB70588b1A51F847d13158ff18E9Cac861dF5Fb00"
                },
                "tokenRegistry": {
                    "implementation": "0xa7E4Fea3c1468D6C1A3A77e21e6e43Daed855C1b",
                    "proxy": "0x0A6f564C5c9BeBD66F1595f1B51D1F3de6Ef3b79",
                    "beacon": "0x4D5ff8A01ed833E11Aba43821D2881A5F2911F98"
                },
                "bridgeToken": {
                    "implementation": "0x4ad6444b55729f657A71a82a5448F85aC8aA47ba",
                    "proxy": "0x9f7eA856bA1fB88d35e000c45E75F134A756Ac4F",
                    "beacon": "0x8ca56E6235D83ff2F4E779F0b35A6c856d5a2fb2"
                },
                "ethHelper": "0x2d6775C1673d4cE55e1f827A0D53e62C43d1F304"
            }
        },
        "moonbeam": {
            "contracts": {
                "bridgeRouter": {
                    "implementation": "0x4D5ff8A01ed833E11Aba43821D2881A5F2911F98",
                    "proxy": "0xD3dfD3eDe74E0DCEBC1AA685e151332857efCe2d",
                    "beacon": "0x0A6f564C5c9BeBD66F1595f1B51D1F3de6Ef3b79"
                },
                "tokenRegistry": {
                    "implementation": "0x8ca56E6235D83ff2F4E779F0b35A6c856d5a2fb2",
                    "proxy": "0xa7E4Fea3c1468D6C1A3A77e21e6e43Daed855C1b",
                    "beacon": "0x9f7eA856bA1fB88d35e000c45E75F134A756Ac4F"
                },
                "bridgeToken": {
                    "implementation": "0xEbB99A5B3021C86301dF241d3A32DBEBa5C15801",
                    "proxy": "0x4ad6444b55729f657A71a82a5448F85aC8aA47ba",
                    "beacon": "0x969d515486Ba6133400aC9C73298586426a090F5"
                },
                "ethHelper": "0xB70588b1A51F847d13158ff18E9Cac861dF5Fb00"
            }
        }
    }
}
yourbuddyconner commented 2 years ago

We should write a tool that validates this config:

ErinHales commented 2 years ago

Looks good. We also have chainID (https://chainlist.org/) and block explorer url in the GUI configs

yourbuddyconner commented 2 years ago

Add a connections: [] field to the bridge config to allow for different connection graphs between bridge and core

yourbuddyconner commented 2 years ago

Take two on unified config, splitting up the files on a logical Home deployment.

https://gist.github.com/yourbuddyconner/f99bb628599a234c1f2d515fe579ed4e

arnaud036 commented 2 years ago

Having a unified file configuration structure consumed by more than one software component (nomad-deploy, nomad-sdk, agent, ....) will introduce some software development and management challenges. I believe, each component should consume configurations from their own config files (i-e: agent.json, nomad-deploy.json,....) with the data structure of their choice. Even if we have duplicated values from one config files to the other, centralizing configuration can be handled at a higher level of our stack instead.

Below are the pros and cons of the current proposal:

Pros:

Cons:

I think a solution would be to start refactoring the configuration structure of each component to easily add or remove a network and separate the component specific config. Once done, we should work on a solution to centralize these configurations by engineering or introducing an existing system.

prestwich commented 2 years ago

Creates a dependency on the configuration data structure across all stack components.

this seems better to me than the current state, where there are >=1 struct per component.

other cons I agree with