Add Docker Compose support

[Menelion]

I'm considering adding Keygen to our small business. I've just read the self-host documentation and, although the solution is based on Docker, there is no mention of how to make an optimal setup with Docker Compose. I.e., you have to have Postgres, Redis and stuff outside Docker or separately add those containers on your own.
Would be nice to have a Compose example if you at Keygen use it yourselves.

[ezekg]

The setup step in particular makes a one-command docker-compose tough. But open to suggestions. Since docker-compose isn't commonly used in production, it isn't a high priority for me to implement myself. I'd of course be open to a PR, though!

[MylesWritesCode]

I'm also wondering this. This is the closest I got, but I'm still working through setting it up in a dev environment. It requires you allowing tty in your compose and using docker compose exec to first set it up, then to re-run the docker compose with a changed command.

[ezekg]

The setup Rake task is mainly an interactive way to seed the database, and create an account and an initial admin. It also makes sure required environment variables are set, such as environment variables for Rails internals and for at-work encryption. But granted you already have all of that, I'm fine with an option to bypass the setup Rake task entirely.

Bypassing the setup sounds like it'd work nicely for docker-compose. I'm thinking maybe add the account and admin creation to db/seeds.rb in the case the KEYGEN_ACCOUNT_ID, KEYGEN_ADMIN_EMAIL and KEYGEN_ADMIN_PASSWORD environment variables are set.

Something like this in db/seeds.rb may work? I haven’t done a whole lot of thinking on it but it seems like it’d work.

# file: db/seeds.rb
# ...

if ENV.key?('KEYGEN_ACCOUNT_ID') && ENV.key?('KEYGEN_ADMIN_EMAIL') && ENV.key?('KEYGEN_ADMIN_PASSWORD')
  id = ENV.fetch('KEYGEN_ACCOUNT_ID')

  Account.find_or_create_by!(id:) do |account|
    email    = ENV.fetch('KEYGEN_ADMIN_EMAIL')
    password = ENV.fetch('KEYGEN_ADMIN_PASSWORD')

    account.assign_attributes(
      billing_attributes: { state: 'subscribed' },
      users_attributes: [{ email:, password: }],
      protected: true,
    )
  end
end

Since rails db:seed needs to be run regardless to seed event types and permissions, I think that's a decent solution. But I honestly haven’t used docker-compose in years, so I’m unsure if this makes 100% sense. Let me know?

[vivekpd15]

Any plans to get this in @ezekg? We want to host this on Kubernetes and automated setup would be extremely helpful.

[ezekg]

No official plans, but open to a PR.

[MShrimp4]

(Reposted since I used the wrong github account)

Copy .env and docker-compose.yml from below and run this command. Initial setup: (make sure to comment out web/worker section)

docker compose up -d
docker run --rm -it \
  --net keygen-net \
  -e SECRET_KEY_BASE="$(openssl rand -hex 64)" \
  -e ENCRYPTION_DETERMINISTIC_KEY="$(openssl rand -base64 32)" \
  -e ENCRYPTION_PRIMARY_KEY="$(openssl rand -base64 32)" \
  -e ENCRYPTION_KEY_DERIVATION_SALT="$(openssl rand -base64 32)" \
  -e DATABASE_URL="postgres://postgres:postgres@keygen-postgres:5432/postgres" \
  -e REDIS_URL="redis://keygen-redis:6379" \
  -e KEYGEN_HOST="api.keygen.localhost" \
  -e KEYGEN_MODE="singleplayer" \
  -e KEYGEN_EDITION="EE" \
  keygen/api setup

The output might be something like:

...
To complete setup, run the following in a shell, or add it to a shell profile:
  export SECRET_KEY_BASE=foo
  export ENCRYPTION_DETERMINISTIC_KEY=bar
  export ENCRYPTION_PRIMARY_KEY=baz
  export ENCRYPTION_KEY_DERIVATION_SALT=salty
  export KEYGEN_EDITION=CE
  export KEYGEN_MODE=singleplayer
  export KEYGEN_ACCOUNT_ID=account-id-gibberish
  export KEYGEN_HOST=domain.example.com
...

Make sure to copy it all (without "export ") into .config/keygen.env, and uncomment web/worker section from docker-compose.yml and re-run.

.env

KEYGEN_WEB_PORT=3000

.config/keygen.env (Put those EXPORTs and other environment variables in here)

SECRET_KEY_BASE=...
ENCRYPTION_DETERMINISTIC_KEY=...
ENCRYPTION_PRIMARY_KEY=...
ENCRYPTION_KEY_DERIVATION_SALT=...
KEYGEN_EDITION=...
KEYGEN_MODE=...
KEYGEN_ACCOUNT_ID=...
KEYGEN_HOST=domain.example.com

DATABASE_URL=postgres://postgres:postgres@keygen-postgres:5432/postgres
REDIS_URL=redis://keygen-redis:6379

docker-compose.yml

version: "3"

services:
  ##Uncomment the following after initial setup & restart
  #worker:
  #  image: "keygen/api"
  #  command: worker
  #  container_name: keygen-worker
  #  restart: unless-stopped
  #  depends_on:
  #    - db
  #    - redis
  #  networks:
  #    - keygen-net
  #  env_file:
  #    - .config/keygen.env
  #  volumes:
  #    - ./keygen:/etc/keygen
  #web:
  #  image: "keygen/api"
  #  command: web
  #  container_name: keygen-web
  #  restart: unless-stopped
  #  depends_on:
  #    - db
  #    - redis
  #  ports:
  #    - "${KEYGEN_WEB_PORT}:3000"
  #  networks:
  #    - keygen-net
  #  env_file:
  #    - .config/keygen.env
  #  volumes:
  #    - ./keygen:/etc/keygen
  redis:
    image: redis
    restart: unless-stopped
    container_name: keygen-redis
    networks:
      - keygen-net
    volumes:
      - ./redis:/data
  db:
    image: postgres
    restart: unless-stopped
    container_name: keygen-postgres
    networks:
      - keygen-net
    environment:
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_USER=postgres
    volumes:
      - ./db:/var/lib/postgresql/data

networks:
  keygen-net:

[derekhe]

I create an example here: https://github.com/derekhe/keygen-docker-compose

[ezekg]

Looping in https://github.com/keygen-sh/community/discussions/77#discussioncomment-7621148:

To skip the setup command, you will need to manually do what the setup command does. But that's hard to do, because it involves creating an account record, which has encrypted values that can't be easily set via raw SQL. So the best way is to run the setup command out-of-band to initialize the database and the account.

You can run the setup command without interaction by supplying the following environment variables during setup:

KEYGEN_ACCOUNT_ID="any UUIDv4"
KEYGEN_ADMIN_EMAIL="some@email.example"
KEYGEN_ADMIN_PASSWORD="secret"

As discussed in keygen-sh/keygen-api#727, I'd be open to a PR that adds account creation to the seed script.

[ezekg]

Also see the notice here: https://keygen.sh/docs/self-hosting/#configure-keygen

The setup command requires a TTY by default. If you're deploying to a platform that does not provide a TTY, you can manually supply the following environment variables to skip all prompts: SECRET_KEY_BASE, ENCRYPTION_DETERMINISTIC_KEY, ENCRYPTION_PRIMARY_KEY, ENCRYPTION_KEY_DERIVATION_SALT, KEYGEN_ACCOUNT_ID, KEYGEN_ADMIN_EMAIL, KEYGEN_ADMIN_PASSWORD, KEYGEN_EDITION, and KEYGEN_MODE.

You can supply all of these env vars during an out-of-band setup.

[ezekg]

/bounty $200

[algora-pbc[bot]]

💎 $200 bounty • Keygen

Steps to solve:

1. Start working: Comment /attempt #727 with your implementation plan
2. Submit work: Create a pull request including /claim #727 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to keygen-sh/keygen-api!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @neo773		#886

[algora-pbc[bot]]

💡 @neo773 submitted a pull request that claims the bounty. You can visit your bounty board to reward.

[ezekg]

We have an official Compose file now: https://github.com/keygen-sh/keygen-api/blob/master/docker-compose.yaml

docker-compose --profile setup run --rm setup
docker-compose up

lmk your thoughts and feedback.

[algora-pbc[bot]]

🎉🎈 @neo773 has been awarded $200! 🎈🎊

• Share on socials
• Give feedback