search

getsentry / self-hosted

Sentry, feature-complete and packaged up for low-volume deployments and proofs-of-concept
https://develop.sentry.dev/self-hosted/
Other
7.89k stars 1.77k forks source link

Vague ConfigurationError, need ideas on how to debug #1217

Closed dwene closed 2 years ago

dwene commented 2 years ago

Version

21.8.0

Steps to Reproduce

I'm getting this very vague configuration error, without a stacktrace or anything, so I don't know to begin debugging this:

!! Configuration error: ConfigurationError("AttributeError: 'NoneType' object has no attribute 'lower'",)

I'm running using version 12.0.0 of the sentry helm chart found here:

https://github.com/sentry-kubernetes/charts/tree/12.0.0/sentry

Here's my configs:

system.admin-email: "derek@lambdacurry.dev"
system.secret-key: "REDACTED"
system.url-prefix: "https://sentry.lambdacurry.dev"

# This URL will be used to tell Symbolicator where to obtain the Sentry source.
# See https://getsentry.github.io/symbolicator/api/
system.internal-url-prefix: 'http://sentry-web:9000'
symbolicator.enabled: false

##########
# Github #
##########

##########
# Google #
##########
auth-google.client-id: "REDACTED"
auth-google.client-secret: "REDACTED"

#########
# Slack #
#########

#########
# Redis #
#########
redis.clusters:
  default:
    hosts:
      0:
        host: "sentry-sentry-redis-master"
        port: 6379
        password: ""

################
# File storage #
################
# Uploaded media uses these `filestore` settings. The available
# backends are either `filesystem` or `s3`.
filestore.backend: "s3"
filestore.options:
  access_key: "REDACTED"
  secret_key: "REDACTED"
  bucket_name: "lambdacurrysentry"
  default_acl: "private"

mail.use-ssl: false
from sentry.conf.server import *  # NOQA
from distutils.util import strtobool

SENTRY_LOG_LEVEL="DEBUG"

DATABASES = {
    "default": {
        "ENGINE": "sentry.db.postgres",
        "NAME": "sentry",
        "USER": "postgres",
        "PASSWORD": os.environ.get("POSTGRES_PASSWORD", "REDACTED"),
        "HOST": "sentry-sentry-postgresql",
        "PORT": 5432,
    }
}

# You should not change this setting after your database has been created
# unless you have altered all schemas first
SENTRY_USE_BIG_INTS = True

###########
# General #
###########

# Instruct Sentry that this install intends to be run by a single organization
# and thus various UI optimizations should be enabled.
SENTRY_SINGLE_ORGANIZATION = True

SENTRY_OPTIONS["system.event-retention-days"] = int(env('SENTRY_EVENT_RETENTION_DAYS') or "90")

#########
# Queue #
#########

# See https://docs.getsentry.com/on-premise/server/queue/ for more
# information on configuring your queue broker and workers. Sentry relies
# on a Python framework called Celery to manage queues.
BROKER_URL = os.environ.get("BROKER_URL", "redis://:@sentry-sentry-redis-master:6379/0")

#########
# Cache #
#########

# Sentry currently utilizes two separate mechanisms. While CACHES is not a
# requirement, it will optimize several high throughput patterns.

# CACHES = {
#     "default": {
#         "BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
#         "LOCATION": ["memcached:11211"],
#         "TIMEOUT": 3600,
#     }
# }

# A primary cache is required for things such as processing events
SENTRY_CACHE = "sentry.cache.redis.RedisCache"

DEFAULT_KAFKA_OPTIONS = {
    "bootstrap.servers": "sentry-kafka:9092",
    "message.max.bytes": 50000000,
    "socket.timeout.ms": 1000,
}

SENTRY_EVENTSTREAM = "sentry.eventstream.kafka.KafkaEventStream"
SENTRY_EVENTSTREAM_OPTIONS = {"producer_configuration": DEFAULT_KAFKA_OPTIONS}

KAFKA_CLUSTERS["default"] = DEFAULT_KAFKA_OPTIONS

###############
# Rate Limits #
###############

# Rate limits apply to notification handlers and are enforced per-project
# automatically.

SENTRY_RATELIMITER = "sentry.ratelimits.redis.RedisRateLimiter"

##################
# Update Buffers #
##################

# Buffers (combined with queueing) act as an intermediate layer between the
# database and the storage API. They will greatly improve efficiency on large
# numbers of the same events being sent to the API in a short amount of time.
# (read: if you send any kind of real data to Sentry, you should enable buffers)

SENTRY_BUFFER = "sentry.buffer.redis.RedisBuffer"

##########
# Quotas #
##########

# Quotas allow you to rate limit individual projects or the Sentry install as
# a whole.

SENTRY_QUOTAS = "sentry.quotas.redis.RedisQuota"

########
# TSDB #
########

# The TSDB is used for building charts as well as making things like per-rate
# alerts possible.

SENTRY_TSDB = "sentry.tsdb.redissnuba.RedisSnubaTSDB"

#########
# SNUBA #
#########

SENTRY_SEARCH = "sentry.search.snuba.EventsDatasetSnubaSearchBackend"
SENTRY_SEARCH_OPTIONS = {}
SENTRY_TAGSTORE_OPTIONS = {}

###########
# Digests #
###########

# The digest backend powers notification summaries.

SENTRY_DIGESTS = "sentry.digests.backends.redis.RedisBackend"

##############
# Web Server #
##############

SENTRY_WEB_HOST = "0.0.0.0"
SENTRY_WEB_PORT = 9000
SENTRY_PUBLIC = False
SENTRY_WEB_OPTIONS = {
    "http": "%s:%s" % (SENTRY_WEB_HOST, SENTRY_WEB_PORT),
    "protocol": "uwsgi",
    # This is needed to prevent https://git.io/fj7Lw
    "uwsgi-socket": None,

    # These ase for proper HTTP/1.1 support from uWSGI
    # Without these it doesn't do keep-alives causing
    # issues with Relay's direct requests.
    "http-keepalive": True,
    "http-chunked-input": True,
    # the number of web workers
    'workers': 3,
    # Turn off memory reporting
    "memory-report": False,
    # Some stuff so uwsgi will cycle workers sensibly
    'max-requests': 100000,
    'max-requests-delta': 500,
    'max-worker-lifetime': 86400,
    # Duplicate options from sentry default just so we don't get
    # bit by sentry changing a default value that we depend on.
    'thunder-lock': True,
    'log-x-forwarded-for': False,
    'buffer-size': 32768,
    'limit-post': 209715200,
    'disable-logging': True,
    'reload-on-rss': 600,
    'ignore-sigpipe': True,
    'ignore-write-errors': True,
    'disable-write-exception': True,
}

###########
# SSL/TLS #
###########

# If you're using a reverse SSL proxy, you should enable the X-Forwarded-Proto
# header and enable the settings below

# SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
# SESSION_COOKIE_SECURE = True
# CSRF_COOKIE_SECURE = True
# SOCIAL_AUTH_REDIRECT_IS_HTTPS = True

# End of SSL/TLS settings

############
# Features #
############

SENTRY_FEATURES = {
  "auth:register": True
}
SENTRY_FEATURES["projects:sample-events"] = False
SENTRY_FEATURES.update(
    {
        feature: True
        for feature in ("organizations:advanced-search",
            "organizations:android-mappings",
            "organizations:api-keys",
            "organizations:boolean-search",
            "organizations:related-events",
            "organizations:alert-filters",
            "organizations:custom-symbol-sources",
            "organizations:dashboards-basic",
            "organizations:dashboards-edit",
            "organizations:data-forwarding",
            "organizations:discover",
            "organizations:discover-basic",
            "organizations:discover-query",
            "organizations:enterprise-perf",
            "organizations:event-attachments",
            "organizations:events",
            "organizations:global-views",
            "organizations:incidents",
            "organizations:metric-alert-builder-aggregate",
            "organizations:metric-alert-gui-filters",
            "organizations:integrations-event-hooks",
            "organizations:integrations-issue-basic",
            "organizations:integrations-issue-sync",
            "organizations:integrations-alert-rule",
            "organizations:integrations-chat-unfurl",
            "organizations:integrations-incident-management",
            "organizations:integrations-ticket-rules",
            "organizations:integrations-vsts-limited-scopes",
            "organizations:integrations-stacktrace-link",
            "organizations:internal-catchall",
            "organizations:invite-members",
            "organizations:large-debug-files",
            "organizations:monitors",
            "organizations:onboarding",
            "organizations:org-saved-searches",
            "organizations:performance-view",
            "organizations:project-detail",
            "organizations:relay",
            "organizations:release-performance-views",
            "organizations:rule-page",
            "organizations:set-grouping-config",
            "organizations:custom-event-title",
            "organizations:slack-migration",
            "organizations:sso-basic",
            "organizations:sso-rippling",
            "organizations:sso-saml2",
            "organizations:sso-migration",
            "organizations:stacktrace-hover-preview",
            "organizations:symbol-sources",
            "organizations:transaction-comparison",
            "organizations:usage-stats-graph",
            "organizations:inbox",
            "organizations:unhandled-issue-flag",
            "organizations:invite-members-rate-limits",
            "organizations:dashboards-v2",

            "projects:alert-filters",
            "projects:custom-inbound-filters",
            "projects:data-forwarding",
            "projects:discard-groups",
            "projects:issue-alerts-targeting",
            "projects:minidump",
            "projects:rate-limits",
            "projects:sample-events",
            "projects:servicehooks",
            "projects:similarity-view",
            "projects:similarity-indexing",
            "projects:similarity-view-v2",
            "projects:similarity-indexing-v2",
            "projects:reprocessing-v2",

            "projects:plugins",
        )
    }
)

#######################
# Email Configuration #
#######################
SENTRY_OPTIONS['mail.backend'] = os.getenv("SENTRY_EMAIL_BACKEND", "smtp")
SENTRY_OPTIONS['mail.use-tls'] = bool(strtobool(os.getenv("SENTRY_EMAIL_USE_TLS", "true")))
SENTRY_OPTIONS['mail.use-ssl'] = bool(strtobool(os.getenv("SENTRY_EMAIL_USE_SSL", )))
SENTRY_OPTIONS['mail.username'] = os.getenv("SENTRY_EMAIL_USERNAME", "REDACTED")
SENTRY_OPTIONS['mail.password'] = os.getenv("SENTRY_EMAIL_PASSWORD", "REDACTED")
SENTRY_OPTIONS['mail.port'] = int(os.getenv("SENTRY_EMAIL_PORT", "587"))
SENTRY_OPTIONS['mail.host'] = os.getenv("SENTRY_EMAIL_HOST", "email-smtp.us-east-1.amazonaws.com")
SENTRY_OPTIONS['mail.from'] = os.getenv("SENTRY_EMAIL_FROM", "sentry@lambdacurry.dev")

#########################
# Bitbucket Integration #
########################

# BITBUCKET_CONSUMER_KEY = 'YOUR_BITBUCKET_CONSUMER_KEY'
# BITBUCKET_CONSUMER_SECRET = 'YOUR_BITBUCKET_CONSUMER_SECRET'

#########
# Relay #
#########
SENTRY_RELAY_WHITELIST_PK = []
SENTRY_RELAY_OPEN_REGISTRATION = True

# No Python Extension Config Given

Expected Result

There would be a stacktrace with the error, so I could at least go find what line it's on.

Actual Result

No stack trace, and there's over 200 references to .lower in the sentry codebase.

chadwhitacre commented 2 years ago

Probably best to ask over at sentry-kubernetes, @dwene. That's a community effort not supported by Sentry.

chadwhitacre commented 2 years ago

If you can repro with stock self-hosted, then this is definitely the place to be. :)

dwene commented 2 years ago

I believe I figured it out, it was this line.

SENTRY_OPTIONS['mail.use-ssl'] = bool(strtobool(os.getenv("SENTRY_EMAIL_USE_SSL", )))

Since the default was null/NoneType that broke the config.

chadwhitacre commented 2 years ago

Glad you got it sorted @dwene, thanks for circling back and leaving a note for others in the future. :)