search

getsentry / self-hosted

Sentry, feature-complete and packaged up for low-volume deployments and proofs-of-concept
https://develop.sentry.dev/self-hosted/
Other
7.94k stars 1.78k forks source link

Cannot run install again on v24.10.0 #3393

Open gander opened 1 month ago

gander commented 1 month ago

Self-Hosted Version

24.10.0

CPU Architecture

x86_64

Docker Version

27.0.3

Docker Compose Version

2.28.1

Steps to Reproduce

Try second run ./install.sh to apply configuration in .env.custom:

SETUP_JS_SDK_ASSETS=1
SETUP_JS_SDK_KEEP_OLD_ASSETS=1

Expected Result

Complete installation

Actual Result

▶ Parsing command line ...

▶ Detecting Docker platform
Detected Docker platform is linux/amd64

▶ Initializing Docker Compose ...

▶ Setting up error handling ...
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 292B 0.0s done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/library/debian:bookworm-slim
#2 DONE 1.0s

#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [1/2] FROM docker.io/library/debian:bookworm-slim@sha256:36e591f228bb9b99348f584e83f16e012c33ba5cad44ef5981a1d7c0a93eca22
#4 resolve docker.io/library/debian:bookworm-slim@sha256:36e591f228bb9b99348f584e83f16e012c33ba5cad44ef5981a1d7c0a93eca22 0.0s done
#4 DONE 0.0s

#5 [2/2] RUN set -x   && apt-get update   && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends jq   && apt-get clean   && rm -rf /var/lib/apt/lists/*
#5 CACHED

#6 exporting to image
#6 exporting layers done
#6 writing image sha256:b0dbe53fc8fa4df30cf3691606b2f7962e5fbc82a2a30e9f26e5e8fd0fc986e3 done
#6 naming to docker.io/library/sentry-self-hosted-jq-local 0.0s done
#6 DONE 0.0s
latest: Pulling from getsentry/sentry-cli
Digest: sha256:9e70afafd91f42bda07b123ac6d8ef23bde1e7088c8b5d947e6a6ce8e00a8402
Status: Image is up to date for getsentry/sentry-cli:latest
docker.io/getsentry/sentry-cli:latest

▶ Checking for latest commit ... 
skipped

▶ Checking minimum requirements ...
Found Docker version 27.0.3
Found Docker Compose version 2.28.1

▶ Upgrading Clickhouse ...
time="2024-10-24T10:17:45+02:00" level=warning msg="The \"HEALTHCHECK_INTERVAL\" variable is not set. Defaulting to a blank string."
time="2024-10-24T10:17:45+02:00" level=warning msg="The \"HEALTHCHECK_TIMEOUT\" variable is not set. Defaulting to a blank string."
time="2024-10-24T10:17:45+02:00" level=warning msg="The \"HEALTHCHECK_RETRIES\" variable is not set. Defaulting to a blank string."
time="2024-10-24T10:17:45+02:00" level=warning msg="The \"SNUBA_IMAGE\" variable is not set. Defaulting to a blank string."
time="2024-10-24T10:17:45+02:00" level=warning msg="The \"SYMBOLICATOR_IMAGE\" variable is not set. Defaulting to a blank string."
time="2024-10-24T10:17:45+02:00" level=warning msg="The \"SNUBA_IMAGE\" variable is not set. Defaulting to a blank string."
time="2024-10-24T10:17:45+02:00" level=warning msg="The \"RELAY_IMAGE\" variable is not set. Defaulting to a blank string."
time="2024-10-24T10:17:45+02:00" level=warning msg="The \"HEALTHCHECK_INTERVAL\" variable is not set. Defaulting to a blank string."
time="2024-10-24T10:17:45+02:00" level=warning msg="The \"HEALTHCHECK_TIMEOUT\" variable is not set. Defaulting to a blank string."
time="2024-10-24T10:17:45+02:00" level=warning msg="The \"HEALTHCHECK_RETRIES\" variable is not set. Defaulting to a blank string."
error while interpolating services.postgres.healthcheck.retries: failed to cast to expected type: strconv.Atoi: parsing "": invalid syntax
Error in install/upgrade-clickhouse.sh:15.
'$dc up -d clickhouse' exited with status 15
-> ./install.sh:main:25
--> install/upgrade-clickhouse.sh:source:15

Envelope from file sentry-envelope-27b1704d8c71ec8d6cdacd9a18c0f35c dispatched
Cleaning up...

Event ID

27b1704d8c71ec8d6cdacd9a18c0f35c

aldy505 commented 1 month ago

Oh you have a malformed value on your docker compose file. Can you share the services -> postgres section?

gander commented 1 month ago

Oh you have a malformed value on your docker compose file. Can you share the services -> postgres section?

  postgres:
    <<: *restart_policy
    # Using the same postgres version as Sentry dev for consistency purposes
    image: "postgres:14.11"
    healthcheck:
      <<: *healthcheck_defaults
      # Using default user "postgres" from sentry/sentry.conf.example.py or value of POSTGRES_USER if provided
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-postgres}"]
    command:
      [
        "postgres",
        "-c",
        "max_connections=${POSTGRES_MAX_CONNECTIONS:-100}",
      ]
    environment:
      POSTGRES_HOST_AUTH_METHOD: "trust"
    volumes:
      - "sentry-postgres:/var/lib/postgresql/data"
gander commented 1 month ago 
x-healthcheck-defaults: &healthcheck_defaults
  # Avoid setting the interval too small, as docker uses much more CPU than one would expect.
  # Related issues:
  # https://github.com/moby/moby/issues/39102
  # https://github.com/moby/moby/issues/39388
  # https://github.com/getsentry/self-hosted/issues/1000
  interval: "$HEALTHCHECK_INTERVAL"
  timeout: "$HEALTHCHECK_TIMEOUT"
  retries: $HEALTHCHECK_RETRIES
  start_period: 10s
gander commented 1 month ago

.env

HEALTHCHECK_INTERVAL=30s
HEALTHCHECK_TIMEOUT=1m30s
HEALTHCHECK_RETRIES=10
gander commented 1 month ago

https://github.com/getsentry/self-hosted/blob/8fd24d02312f9fd7990c1ad0808d561c7b4f80b5/install/_lib.sh#L11-L17

seriously? when any documentation says that some custom .env.xyz override .env it doesn't mean that it replaces it completely, it just gives the effect of merging the values ​​from .env and .env.xyz!

aldy505 commented 1 month ago

Do you have any .env.custom file? The values should exist there too.

gander commented 1 month ago

It is logical and commonly used that .env.* inherits .env. Any other solution, which additionally appears in the documentation in two mutually exclusive forms, only introduces confusion and problems.

  1. Environment variables—The available keys are defined in .env. Use some system-dependent means of setting environment variables if you need to override any of them. To avoid Git changes, simply create a file called .env.custom and insert your system-dependent environment variables there. In order to use this, please use docker compose --env-file /path/to/.env.custom up -d.

https://develop.sentry.dev/self-hosted/#customize-dotenv-env-file

gander commented 1 month ago

previous:

SETUP_JS_SDK_ASSETS=1
SETUP_JS_SDK_KEEP_OLD_ASSETS=1

current:

COMPOSE_PROJECT_NAME=sentry-self-hosted
COMPOSE_PROFILES=feature-complete
SENTRY_EVENT_RETENTION_DAYS=90
# You can either use a port number or an IP:PORT combo for SENTRY_BIND
# See https://docs.docker.com/compose/compose-file/#ports for more
SENTRY_BIND=9000
# Set SENTRY_MAIL_HOST to a valid FQDN (host/domain name) to be able to send emails!
# SENTRY_MAIL_HOST=example.com
SENTRY_IMAGE=getsentry/sentry:24.10.0
SNUBA_IMAGE=getsentry/snuba:24.10.0
RELAY_IMAGE=getsentry/relay:24.10.0
SYMBOLICATOR_IMAGE=getsentry/symbolicator:24.10.0
VROOM_IMAGE=getsentry/vroom:24.10.0
HEALTHCHECK_INTERVAL=30s
HEALTHCHECK_TIMEOUT=1m30s
HEALTHCHECK_RETRIES=10
# Caution: Raising max connections of postgres increases CPU and RAM usage
# see https://github.com/getsentry/self-hosted/pull/2740 for more information
POSTGRES_MAX_CONNECTIONS=100
# Set SETUP_JS_SDK_ASSETS to 1 to enable the setup of JS SDK assets
# SETUP_JS_SDK_ASSETS=1

SETUP_JS_SDK_ASSETS=1
SETUP_JS_SDK_KEEP_OLD_ASSETS=1
gander commented 1 month ago

Eventually I realized there was a logical error in the documentation and I guessed that I needed to have everything that is .env also in .env.custom, and after this update the installation was successful.

aldy505 commented 1 month ago

It is logical and commonly used that .env.* inherits .env. Any other solution, which additionally appears in the documentation in two mutually exclusive forms, only introduces confusion and problems.

Sadly Docker does not support that.

gander commented 1 month ago

Sadly Docker does not support that.

You have outdated information

.env

FOO="foo from .env"
FOOBAR="foobar form .env"

.env.override

BAR="bar from .env.override"
FOOBAR="foobar from .env.override"

compose.yaml

services:
    app:
        image: alpine:latest
        command: printenv
        env_file:
            - .env
            - .env.override
docker run --env-file .env --env-file .env.override alpine printenv

Result in both:

FOO="foo from .env"
FOOBAR="foobar from .env.override"
BAR="bar from .env.override"
aldy505 commented 1 month ago

Ah, I was thinking of specifying one --env-file flag. Whereas you're supplying more than one flag. We'll update the docs.

aldy505 commented 1 week ago

PR to the updated docs https://github.com/getsentry/sentry-docs/pull/11807