search

fission-codes / fission

Fission CLI & server
https://runfission.com/docs
119 stars 14 forks source link

Add secp256k1 key support for did:key #625

Closed icidasset closed 2 years ago

icidasset commented 2 years ago

This PR adds secp256k1 key support for did:key, the type of key used by Ethereum, Bitcoin and various other blockchains. We intend to use this initially for wallet auth, where we look up the public key of an Ethereum account and then construct a DID from that client side. In addition to that, we will also sign UCANs with those DIDs, which this PR also adds support for.

Public key types

This PR changes how we store public keys in the database. Before we stored the public key bytes formatted as base64. This is now wrapped in a JSON object, along with the type of the public key. That affects the Update Public Key and Update Public Key via Email Challenge API endpoints for which I created new V3 routes.

New format:

{"key":"Hv+AVRD2WUjUFOsSNbsmrp9fokuwrUnjBcr92f0kxw4=","type":"Ed25519"}
{"key":"AvuqnqcYF49vlzMHKKAuaXjEOaqs/x9V1GnY+rJxx6Y2","type":"Secp256k1"}

Migration

I've written a DB migration that converts the old string format into the json-object format. This can be found at fission-web-server/sql/1658241548_set_public_key_type.sql You can run this migration multiple times, it detects both the old and new format.

SECP256K1 UCANs

UCANs with secp256k1 signatures can be created by using the ES256K algorithm (alg field in the header). This alg value is specified here: https://datatracker.ietf.org/doc/html/rfc8812#section-3.2 Not sure that's the correct spec, but it's the only one I found.

Example: https://github.com/fission-codes/webnative-walletauth/blob/375c63881a300442ec3112bf6a7863fc22ecbb4a/src/webnative.ts#L155-L159

Ethereum signatures

This is currently specifically tuned to Ethereum. Normally the content that one would sign for a UCAN signature would be ${UCAN_HEADER}.${UCAN_PAYLOAD}. But here this content will be:

const regularContentBytes =
  uint8arrays.fromString("${UCAN_HEADER}.${UCAN_PAYLOAD}", "utf8")

const actualContent = keccak_256(
  uint8arrays.concat([
    uint8arrays.fromString("\x19Ethereum Signed Message:\n${regularContentBytes.length}", "utf8"),
    regularContentBytes
  ]) 
)

I made a module that is responsible for "wrapping" this data at Web.UCAN.Signature.Secp256k1.Ethereum.

I guess to see if it's an Ethereum signatures we could detect if the recovery value v of the secp256k1 signature is bigger than or equal to 27 (specification: https://github.com/ethereum/EIPs/blob/master/EIPS/eip-155.md#specification). Although I guess Bitcoin has a v value of 27 as well. Thoughts? Added an Ethereum check which looks for a v value of >= 35.

icidasset commented 2 years ago

@walkah Do I have to do anything special here for Nix and the Github actions? I added a C library https://github.com/bitcoin-core/secp256k1 using Nix (pkgs.secp256k1). Locally I had to add pkgs.pkg-config to get it to work, but I can't get it to work for Github actions. Any thoughts?

Resources: https://nixos.wiki/wiki/FAQ/I_installed_a_library_but_my_compiler_is_not_finding_it._Why%3F

icidasset commented 2 years ago

The error I'm talking about: https://github.com/fission-codes/fission/runs/7582732721?check_suite_focus=true#step:5:12643

icidasset commented 2 years ago

Still can't figure out why data-root lookups are returning a 404 for me with my local instance. If anyone else is trying it out. You can test it with walletauth, by putting the following code in the index.ts file (replacing the existing async function):

;(async () => {
  wn.setup.endpoints({
    api: "http://runfission.test:1337",
    lobby: "http://localhost:8001",
    user: "fissionuser.test"
  })

  webnative.login()
})()
avivash commented 2 years ago

Still can't figure out why data-root lookups are returning a 404 for me with my local instance. If anyone else is trying it out. You can test it with walletauth, by putting the following code in the index.ts file (replacing the existing async function):

;(async () => {
  wn.setup.endpoints({
    api: "http://runfission.test:1337",
    lobby: "http://localhost:8001",
    user: "fissionuser.test"
  })

  webnative.login()
})()

I don't have a huge amount of context around this, but I just installed nix (i know i'm late to the nix party 😅), got stack setup and i successfully ran stack build on your branch, but nothing seems to be running on port 8001. Is there a specific package I need to run? This is my first time using haskell 🙈

bgins commented 2 years ago

I don't have a huge amount of context around this, but I just installed nix (i know i'm late to the nix party 😅), got stack setup and i successfully ran stack build on your branch, but nothing seems to be running on port 8001. Is there a specific package I need to run? This is my first time using haskell 🙈

Hey @avivash! I can give you some pointers on project setup and running the server.

First, a question -- are you using the nix shell in the project? You can use it by running nix-shell at the root of the project. That will take a while to run because it will retrieve and compile a whole lot of Haskell.

Once that finishes and you are in the nix shell, you should be able to run helpme to get a help menu that looks like this:

$ helpme
  ____                                          _     
 / ___|___  _ __ ___  _ __ ___   __ _ _ __   __| |___ 
| |   / _ \| '_ ` _ \| '_ ` _ \ / _` | '_ \ / _` / __|
| |__| (_) | | | | | | | | | | | (_| | | | | (_| \__ \
 \____\___/|_| |_| |_|_| |_| |_|\__,_|_| |_|\__,_|___/

build          | Build entire project
cli-install    | Install the Fission CLI
quality        | Run the complete test suite
repl           | Enter the project REPL
server-debug   | Run the Fission Server in debug verbose mode
server-install | Install the Fission Server
server-start   | Run the currently installed Fission Server
server-update  | Update & run the current server to the latest on the current branch
ssh-prod       | SSH into the production environment
ssh-staging    | SSH into the staging environment
watch          | Autobuild with file watcher
welcome        | Print the pretty welcome

The build command is a wrapper around stack build that builds all of the projects (CLI, server, etc). server-install will install the server binary where you can run it, and server-debug will run the server with some logging.

Before you run the server, there's a bit of docker, server and DNS configuration that is documented here: https://github.com/fission-codes/fission/tree/main/fission-web-server#using-docker.

I think that should be enough to get you up and running, but let me know if you'd like a sync chat to help out with any of it.

avivash commented 2 years ago

I don't have a huge amount of context around this, but I just installed nix (i know i'm late to the nix party 😅), got stack setup and i successfully ran stack build on your branch, but nothing seems to be running on port 8001. Is there a specific package I need to run? This is my first time using haskell 🙈

Hey @avivash! I can give you some pointers on project setup and running the server.

First, a question -- are you using the nix shell in the project? You can use it by running nix-shell at the root of the project. That will take a while to run because it will retrieve and compile a whole lot of Haskell.

Once that finishes and you are in the nix shell, you should be able to run helpme to get a help menu that looks like this:

$ helpme
  ____                                          _     
 / ___|___  _ __ ___  _ __ ___   __ _ _ __   __| |___ 
| |   / _ \| '_ ` _ \| '_ ` _ \ / _` | '_ \ / _` / __|
| |__| (_) | | | | | | | | | | | (_| | | | | (_| \__ \
 \____\___/|_| |_| |_|_| |_| |_|\__,_|_| |_|\__,_|___/

build          | Build entire project
cli-install    | Install the Fission CLI
quality        | Run the complete test suite
repl           | Enter the project REPL
server-debug   | Run the Fission Server in debug verbose mode
server-install | Install the Fission Server
server-start   | Run the currently installed Fission Server
server-update  | Update & run the current server to the latest on the current branch
ssh-prod       | SSH into the production environment
ssh-staging    | SSH into the staging environment
watch          | Autobuild with file watcher
welcome        | Print the pretty welcome

The build command is a wrapper around stack build that builds all of the projects (CLI, server, etc). server-install will install the server binary where you can run it, and server-debug will run the server with some logging.

Before you run the server, there's a bit of docker, server and DNS configuration that is documented here: https://github.com/fission-codes/fission/tree/main/fission-web-server#using-docker.

I think that should be enough to get you up and running, but let me know if you'd like a sync chat to help out with any of it.

Amazing! Thanks @bgins! I'll give that a try 🙏🏼

bgins commented 2 years ago

Still can't figure out why data-root lookups are returning a 404 for me with my local instance. If anyone else is trying it out. You can test it with walletauth, by putting the following code in the index.ts file (replacing the existing async function):

;(async () => {
  wn.setup.endpoints({
    api: "http://runfission.test:1337",
    lobby: "http://localhost:8001",
    user: "fissionuser.test"
  })

  webnative.login()
})()

I also ran into these issues, making calls from Insomnia. Seems to be down where we are calling on PowerDNS to resolve DNS records. Do we know if this is an issue in outside of this PR? If so, maybe we could test it on a pizza instance? Or move forward and test this part of the functionality on staging?

walkah commented 2 years ago

I also ran into these issues, making calls from Insomnia. Seems to be down where we are calling on PowerDNS to resolve DNS records. Do we know if this is an issue in outside of this PR? If so, maybe we could test it on a pizza instance? Or move forward and test this part of the functionality on staging?

If it's working locally - and tests are all passing - I'd vote to merge and deploy to staging for additional testing (and open subsequent issues if stuff comes up on staging).

icidasset commented 2 years ago

@bgins

I tested the following

Awesome thanks!

One thing that I tried that failed was updating a user public key from Ed25519 to secp256k1. Not sure if that is a feature that we want.

Hmm, that should be possible. How did it fail?

A question about the old style public keys. Are those a compatibility layer while we migrate the database? Do we keep them around to support v2 routes?

Indeed, v2 routes still work with the old format. No database migration needed for those, they're automatically converted to the new style behind the scenes. Just happens to only support ED & RSA keys because of how we used to discerned those two.

bgins commented 2 years ago

One thing that I tried that failed was updating a user public key from Ed25519 to secp256k1. Not sure if that is a feature that we want.

Hmm, that should be possible. How did it fail?

It fails with a 400 with this message in the response

Error in $: Unable to decode Ed25519 PK because: CryptoFailed CryptoError_PublicKeySizeInvalid / BDTfGgRGmbA1Vjob/f8GU2ZjvY3KYP6f2DMor5Z7Te33wWPD48ik4nQeD7Qez549ue78xK5EBohBFkTogZeJ7bo=

The original key that was to be replaced was an Ed25519 key.

icidasset commented 2 years ago

Since updating public keys of the same type works I'm going to merge this in. I have to test transitioning between key types more extensively, but we can do that later (tracking in #629).