Plugin Server Fails to Start on Fresh Self-Hosted Docker Install (Ubuntu 24.04)

Environment

PostHog Version: latest
Installation Method: Self-hosted Docker
Operating System: Ubuntu 24.04

Description

After performing a fresh installation of PostHog using Docker on Ubuntu 24.04, the plugin server is failing to start. The error logs indicate that the encryption keys are not set, causing a startup failure.

Steps to Reproduce

Set up a fresh Ubuntu 24.04 server
Install Docker and Docker Compose
Follow the PostHog self-hosted Docker installation instructions
Attempt to start the PostHog services

Error Logs

{"level":"error","time":1727979202583,"pid":73,"hostname":"f31d1dde2c9f","error":"Error: Encryption keys are not set\n    at new EncryptedFields (/code/plugin-server/dist/cdp/encryption-utils.js:10:19)\n    at createHub (/code/plugin-server/dist/utils/db/hub.js:184:26)\n    at async setupHub (/code/plugin-server/dist/main/pluginsServer.js:157:34)\n    at async startPluginsServer (/code/plugin-server/dist/main/pluginsServer.js:166:25)","msg":"[MAIN] 💥 Launchpad failure!"}
{"level":"error","time":1727979202583,"pid":73,"hostname":"f31d1dde2c9f","error":{},"msg":"[MAIN] 💥 Exception while starting server, shutting down!"}

Expected Behavior

The plugin server should start successfully without any errors.

Actual Behavior

The plugin server fails to start, reporting that encryption keys are not set.

Additional Context

This is a fresh installation, so no custom configurations have been applied.
All other services (ClickHouse, Kafka, PostgreSQL, Redis, and object storage) appear to connect successfully.

Questions

Are there any additional configuration steps required for setting encryption keys in a self-hosted Docker environment?
Could this be related to any recent changes in PostHog or specific requirements for Ubuntu 24.04?

Any assistance in resolving this issue would be greatly appreciated. Thank you!

Debug info

No response

samuelnygaard commented 3 days ago

I also have this issue

Kvintus commented 3 days ago

Same

Kvintus commented 3 days ago

@switchp0rt @samuelnygaard

So... I found the issue. It's not documented anywhere but we have to set the ENCRYPTION_SALT_KEYS env variable to the compose services, it seems like, anywhere where the SECRET_KEY is provided as well. Most likely just the plugin-server .

The value 00beef0000beef0000beef0000beef00 should be fine. 😊

TLDR: set this env var: ENCRYPTION_SALT_KEYS=00beef0000beef0000beef0000beef00

And to someone from the PostHog team, probably @benjackwhite :

The issue originates here: https://github.com/PostHog/posthog/pull/24966

And although it's stated here, that default value should be provided for hobby deployments:

https://github.com/PostHog/posthog/blob/914ef395c85f31c662188268a39df8d883cf7166/posthog/settings/access.py#L74-L75

The same does not seem to be the case for the plugin-server: https://github.com/PostHog/posthog/blob/914ef395c85f31c662188268a39df8d883cf7166/plugin-server/src/config/config.ts#L178-L180

PostHog / posthog