VolumeFi / paloma

The fast blockchain messenger protocol
Apache License 2.0
3 stars 2 forks source link

DEVOPS: Resize and Volume Tasks for State exports and transfer #1459

Closed taariq closed 6 months ago

taariq commented 7 months ago

@rootedbox we need to resize our validator infrastructure. @byte-bandit will you check these requirements and comment.

    • [ ] Resize and attach the validator-mainnet volume to 1TB. This so we can have enough space to export state on the mainnet validator, but on the storage volume as needed during emergencies.
    • [ ] Rename the labels on the volumes consistently

    • lcd-mainnet (1TB)

    • validator-mainnet (1 TB)

    • snapshot-mainnet (1 TB)

    • [ ] Discard all unused volumes. lcd-mainnet at 200GB appears unused.
    • [ ] Do the same for testnet, we have the following servers that we want to keep

    • birdlady-testnet

    • birdpoop2-testnet

    • farfly-testnet

    • feather-testnet

        • [ ] Add farfly-testnet and feather-testnet to the bastion host ssh setup
taariq commented 7 months ago

@verabehr I think we need the same for testnet?

rootedbox commented 7 months ago

bastion host ssh setup add to your ~/.ssh/config

Host testnet-farfly
    HostName 172.234.229.157
    Port 25022
    User root
    ProxyCommand ssh bastion-host -W %h:%p

Host testnet-feather
    HostName 74.207.234.56
    Port 25022
    User root
    ProxyCommand ssh bastion-host -W %h:%p

cc: @verabehr @taariq

taariq commented 6 months ago

@rootedbox this is not complete. please properly label big as that name is not in the format.

byte-bandit commented 6 months ago

LGTM. I was tempted to ask for identical setups on the nodes as well (OS, users, etc..), but I've come to the conclusion that this diversification is actually beneficial to testing our setup. We'll catch more edge cases this way.

byte-bandit commented 6 months ago

Updated https://www.notion.so/volumefi/SSHing-into-Volume-Paloma-Servers-ff97b346f75d45d3a170549ee023238f to reflect the config changes.

rootedbox commented 6 months ago

I've renamed and setup lcd to be like the other servers lcd-mainnet.. both @verabehr and I agree that testate doesn't need large expensive volumes because it's a test net that won't be big in size.. and because it's a test net we can always reset the network.