Better PGP support (HSM, smartcards, tpm, gpg-agent, etc)

[dumbasPL]

The current implementation uses a basic sequoia_openpgp implementation with a key file and an optional password. I want to rework this to use sequoia_keystore, which supports multiple backends.

This has a few advantages:

• The key material can be stored outside of the container, even if stored as a file on disk (gpg-agent running on host or even sequoia-keystore-server)
• The key material can be stored on a smartcard (Like a YubiKey) or in the TPM (most modern computers have one, and when running in a VM the key can be stored outside of the VM when using a virtual TPM)

In my opinion, we can go one of two routes.

1. support sequoia-keystore-softkeys (for backward compatibility, no additional services required) + sequoia-keystore-gpg-agent (the battle-tested gpg-agent running externally can handle all the implementation details for us)

2. same as 1 + all the other backends to allow running with minimal dependencies on the host at the cost of more maintenance and more dependencies in the container

I would go with 1. I'm willing to make a PR for this. Let me know what you think.

[VirtCode]

This is a cool idea, and I also think number 1 is the way to go.

• Implementing the current behaviour via softkey is good, so we can handle all keys through seqoia-keystore. I really want to preserve the current functionality, as this makes package signing easy to quickly set up, and I deem it sufficiently secure for many users (assuming they don't use their personal keys, as recommended), as there are many more severe risks when running an aur build server.
• I also think implementing the gpg-agent backend to "offload" the other implementations on the gpg-agent is a cool idea, as this opens a ton of possibilities, without us implementing anything extra. If someone needs another backend not possible via gpg-agent (if there are any) and has a valid reason, we can consider that then.

I don't know too much about the seqoia_keystore, but I think you can run it as a separate program/daemon, or as a library. I would suggest implementing everything here as a library, so we don't have to deal with multiple processes etc. inside the container.

If you have any questions regarding the codebase, don't hesitate to ask me or @WhySoBad (who has done the current signing implementation).

[dumbasPL]

If you have any questions regarding the codebase, don't hesitate to ask me

Is there a good place for this? Or do you just want to keep everything here?

A development quick start guide would be nice. A very basic "how to set up a minimal local dev environment". I'm familiar with Rust, but not the entire ecosystem.

[VirtCode]

We can do it here, via discord, or matrix, whatever you prefer.

• My discord would be @virtinstance
• My matrix @virtinstance:matrix.org

As for a quick start guide: The project uses the normal rust project architecture, so everything is managed with cargo. You can open the root of the repo in your code-editor or ide of choice. Make sure you have installed and activated the nightly toolchain with e.g. rustup, because we use some features which are only available there.

To be able to build the server, you need to have a database with the migrations ready and an env set. We'll be using the app directory in the repo root as our testing ground, so set the following env for your terminal session:

DATABASE_URL=sqlite://app/serene.db

If your lsp shows a couple of errors regarding the database file, remember to also set this env for the lsp.

With this env ready, we should initialize the database. For that, we'll need the sqlx-cli, which you can install via cargo:

cargo install sqlx-cli

To create and migrate a database, run the following in your repository root (with the DATABASE_URL env set):

cargo sqlx database setup --source server/migrations

With the database created and migrations applied, you should now be able to build the server. The build will fail if the DATABASE_URL env is not set, or the database it points to is non-existent or has no migrations applied. As we have created the serene.db in the app directory, we'll be running our server there. So to build and run a debug build, head into there and run:

cargo run --bin serene

Yes, I'll document that properly somewhere in the docs sometime in the future.

Some things when running your server: To have a good experience running the server locally, it is recommended to set a few envs when launching it. (These are basically the same as you'd set on the container). On my end, I have:

RUST_LOG=info
PORT=8080 # we can't use 80 because privileges
OWN_REPOSITORY=<your upstream hosted server> # easiest, unless you want to spend the time to figure out the address the build containers can reach your host with

Don't forget to create an authorized_secrets file in the app dir and add your secret, if you want to do anything with the local server.

Also remember that you should have a docker daemon started, to build anything.

To test the local server, you can use the serene cli you already have installed. You can use the --server option to override the server. For example you can do something like this:

serene --server http://localhost:8080 list

As for the codebase, it is generally structured like this:

• the code for the server is in server
• server/data is a module that is shared between the server and cli
• stuff for the runner docker image is in runner
• the cli is in cli