Support and document secure key management for deposits and withdrawals

[torfbolt]

As far as I can tell, the private keys of both the Eth1 node account wallet and the Eth2 HD wallet are currently meant to be stored on the same machine that runs the smartnode. For testnets that is an acceptable convenience feature. However, it runs against all security recommendations for the Eth2 mainnet. So IMO before mainnet launch there should be a well documented way to safely setup a RP smartnode, while keeping the seed / mnemonic airgapped and later in cold storage.

I think node operators can not be expected to operate a mainnet node with the keys protected only by a passphrase on the staking machine itself.

[moles1]

• The wallet mnemonic is never stored on disk, it is simply printed to stdout and then the user is asked to confirm it; once they do the terminal output is cleared. The mnemonic could be generated on an airgapped machine and then the node operator could "restore" their wallet using the same mnemonic on the validating machine, but in practice this is no different.

• Validator keys are hot, so the private keys must be stored on the server performing validation; this is an unavoidable aspect of running an eth2 validator (for everyone, not just RP node operators).

• The node account also needs to be hot since staking minipools is done automatically by the smart node daemon (once a minipool has received user ETH and is ready to stake). nETH, RPL staking rewards, and (occasionally) ETH may be transferred to the node operator, so there is an open issue (https://github.com/rocket-pool/rocketpool/issues/184) to implement a separate "withdrawal address" for these funds to be sent to. Any ETH / RPL moving through the node address for staking will only sit in the node account temporarily.

[moles1]

I think node operators can not be expected to operate a mainnet node with the keys protected only by a passphrase on the staking machine itself.

The node operator is responsible for hardening their own validator nodes - their validators should be protected by additional measures, but that's their prerogative. Validators should not be run on shared machines, and should have all incoming ports closed where possible (e.g. only 22 open) and require SSH pubkey authentication with a passphrase.

[torfbolt]

@moles1 Thanks for the explanations. So let's assume I have a running RP node and want to generate an additional minipool. How is the deposit transaction for this generated, if the node software doesn't have the eth2 seed saved? Or does RP not use a HD wallet for eth2 addresses?

If I understand correctly, starting minipools is the only operation that requires the hot node account. All other actions or transfers from the account are operator-initiated anyway and could e.g. query a passphrase.

Would it be possible to make an option for queuing minipool operations, notifying the operator and doing them once the passphrase is supplied?

I'm just trying to think of ways on how to reduce the attack surface for draining the whole account. Of course validator keys always need to be hot, but those do not carry the risk of losing the whole stake to an attacker.

[torfbolt]

I would like to bump this again with respect to the upcoming mainnet launch. Did the audits yield any findings or recommendations on how to reduce the risk of lost node operator funds due to an attacker obtaining the node hot wallet?

As I understand the situation right now, the attacker in this case could export the key, withdraw all of the rewards in the node account, and also exit all minipools and withdraw the staked ETH in the form of nETH to their own wallet, plus all of the staked RPL after exiting. The node operator could do nothing against this except trying to automate the same withdrawal actions at the earliest possible block, with high gas prices in a race against the attacker.

This is way worse than the case of stolen validator hot keys for a solo staking user that gets their machine hacked. Consequently, RP makes a (IMO unnecessarily) large honeypot out of every node.

Concerning a quick fix/workaround: would it be possible to run the beacon node and validators on a different machine than the smartnode service, once all of the validator keys are generated and the minipools are active? I noticed that the only regular on-chain interaction is the RPL reward claiming. Can this be postponed or done only from time to time on a normally offline machine?

[darcius]

Hey @torfbolt! We will have some news out about this soon, one of the main things we are looking at is baking in the nodes current withdrawal address into any minipools created. This way the only way to withdraw validator funds is using that account which could be a cold wallet Ledger etc.

As far as the RPL claiming, this will need to be done once a month on mainnet and can also go to the set node withdrawal address too.

[torfbolt]

Hey @darcius, thanks for addressing this. I can see that the baked in (on minipool or node level) withdrawal address alleviates most of the problem. I would argue for freezing on node level, as this also solves the risk for the staked RPL.

This only leaves one major point open: if I have a node that's been running for some time, an attacker could have silently grabbed the node hot wallet. Adding a new minipool is only possible by transferring the funds to the node account first, where they would be instantly stolen in this scenario. This is definitely less likely than the other scenarios, but I can't think of a good way around this problem right now. Maybe allow creation of the minipool smart contract without transferring any funds, and showing the address so the user can directly deposit into it from their cold wallet.

[torfbolt]

Another idea regarding withdrawal address: maybe we could use an ephemeral second account derived from the mnemonic, that's never saved to disk, to set/change the withdrawal address. That way, the owner would always be in control, as long as they have their mnemonic. But an attacker with only the first (hot) wallet wouldn't be able to change the withdrawal address.

[lodotek]

Any updates here?

[lodotek]

Wow, This is still open all these years later!