Understand the work required for HSM support with Ethereum and break into tickets

[gordsport]

As part of the Sweeterwater 💦 planning cycle the validators have asked for Hardware Security Module (HSM) support for Ethereum keys

This ticket is for the core team to understand and find a technical solution for this that can be broken into tickets and implemented via the usual core team process

The tasks to complete this refinement are likely to look something like this (but feel free to amend to suit):

• [ ] Discuss and agreed the best way to break the work down into feature tickets (into it's smallest valuable work items)
• [ ] create a set of coherent feature tickets for the work that can be understood (and implemented) by the whole team and prioritised for development in our Backlog.

To include:

• [ ] Co-create feature test scenarios with @cpuja84
• [ ] Understand if and what system tests will be required to be implemented by @vega-paul
• [ ] Create tickets using the new feature ticket template
• [ ] Add tickets to the HSM project board add link to feature in the Ready to Work on column in priority order
• [ ] Review tickets with the team
• [ ] Create the outline of a spec document for review with (and completion by) protocol-design team - if required

[jeremyletang]

Couple of things I can think of that we may want to look at / know answer for at the end of tis spike:

• what tool can use to support hsm (most likely something like clef)
• how does that work
  • is that an API?
  • how do we store credential for those
• how can we test this (software version as you mentionned, but would be good to have real hardware testing done).
• can we use hsm providers as a services for our testnet, what are the costs etc.
• What will be the impact on the codebase? (refactor of the nodewallet).
• can we keep supporting easily the current wallet with software signing + hsm support (do we need to or not?)
• Try to get an undersanding of he performance impact? (Latency introduced, try to get a feel of that with a real network usage).

Also leett's keep in mind that we want to support HSM only for ethereum at first, and that most likely we will want to use a tool like clef which allow management of accounts, with different backends (keystore, hsm, ledger, etc...)

[karlem]

Update on questions from above

• What tool can use to support HSM (most likely something like clef) - Clef can be a useful option for Ethereum

• How does that work - Clef daemon is required to be run. It can be run locally, on a remote server or USB armory

• is that an API? Yes - it has a JSON RPC API that can be used with existing Ethereum client (github.com/ethereum/go-ethereum/rpc) or without.

• How do we store credentials for those? - Credentials aka passwords are store inside Clef. User has to approve every API call either manually or write a custom rule (http://adam.schmideg.net/go-ethereum/clef/Rules)

• How can we test this? - During the integration test, we can start an instance of Clef on AWS or GCP machine. It doesn’t actually have to be a physical device as the API is exactly the same.

• Can we use HSM providers as a service for our Testnet, what are the costs etc. - In terms of Clef we can just deploy an instance of it. The cost is the cost of the machine that Clef would be running on.

• What will be the impact on the codebase? (refactor of the node wallet). - Wallet could use the RPC API to manage these accounts.

• Can we keep supporting easily the current wallet with software signing + HSM support (do we need to or not?) - Question for Valentin.

• Try to get an understanding of the performance impact? (Latency introduced, try to get a feel of that with real network usage). - This all depends on how the validators decide to run their instance of Clef. Whether is a remote server or locally?

[karlem]

Update no. 2

As of today, there is currently multiple options for accounts management backends on Ethereum

• Local key store
• USB Ledger hardware wallet
• USB Trezor hardware wallet
• Smart card wallet
• External signer - Clef is a good example.

There is also a fork of the project that allows direct usage of HSM devices with PKCS 11 interface. Please see here: https://github.com/ThalesGroup/go-ethereum/tree/hsm_wallet_integration_v1.8.13/accounts/hsmwallet

All these options above can be easily plugged into the general Ethereum account manager being used currently by Vega.

[jeremyletang]

Action following the recent meeting:

• @ValentinTrinque @karlem @wwestgarth to discuss how to integrate clef in the nodewallet
• Same persons to discuss refactoring the nodewallet
• Same persons to discuss how to support both clef and raw keystore by the nodewallet to easy testing / running a node locally
• @karlem to join the validators discord to get more information on how the validators are expecting to run the clef.

To be discussed later

• how involve QA with testing clef implementatin.

[wwestgarth]

Just from a security point of view adding support for clef will create a small attack surface to the vega wallet. Clef has a fairly well-defined Security Model about how a sign request coming in is untrusted, until manual invention (or a rule) says it is fine. Its very clear that on the untrusted channel there is no mutual authentication. So this also applies to the Vega-wallets -- it will have no idea what it is sending sign-requests to.

Assuming we are talking to a malicious clef it can really only affect the vega-wallet by:

• Introducing artificial latency
• Returning an invalid signature
• Sending some intentional junk back to try and cause a memory violation (golang has our back)

So it's not too bad and any attack will likely only at most cause a DoS, but it is worth being aware of. I think any attempts at mitigation e.g PIDs/user checks for pipe/port squatting when clef is local, early signature verification if we have the public-key to hand (which is in essence half of a TLS handshake) will be more work for little gain.

As an aside: clef's use use of "rules" to provide automatic signing invalidates their entire security model. I can't see a sufficient way to constrain a rule enough to provide adequate security, but it seems to be a known issues with a mention in the README to add a handshake and make "the addition of rules less dangerous."

Ultimately how the rule is set is a decision for the validator. As I understand the ones asking for HSM support are already using clef so should be aware nuances and will hopefully have their own defence-in-depth strategy.

[jeremyletang]

@wwestgarth the vega wallet will not use clef. Clef will be used only for the purpose of storing ethereum keys.

You assumption are all rights, but we would expect the validators to be first non malicious (2+1 / 3 non malicious at least, so the network keeps creating blocks) but also have secured appropriately their infrastructures.

As you mention yea, the validators are asking for clef, and are used to it, also yea it is for them to make sure they set it up appropriately.

[gordsport]

@vega-paul the broken out tickets for you to review on this are: Add support for Clef to Ethereum node wallet Refactor node wallet to support Clef

Once done and no other tickets are required this can be moved to done so work can be tracked in the child tickets