search

sentry-kubernetes / charts

Easily deploy Sentry on your Kubernetes Cluster
MIT License
1.06k stars 504 forks source link

Ocurred Error: Organization by slug [None] not found. #1391

Open choi2601 opened 1 month ago

choi2601 commented 1 month ago

I encountered the error Organization by slug [None] not found on my self-hosted Sentry instance, and while reviewing the error guide, I saw the message There was an error loading data.

스크린샷 2024-08-19 오후 4 32 21

Here is the relevant portion of my Sentry web's yaml file:

  env:
    - name: SNUBA
      value: http://sentry-snuba:1218
    - name: VROOM
      value: http://sentry-vroom:8085
    - name: SENTRY_SECRET_KEY
      valueFrom:
        secretKeyRef:
          key: key
          name: sentry-sentry-secret
    - name: POSTGRES_PASSWORD
      valueFrom:
        secretKeyRef:
          key: postgres-password
          name: sentry-sentry-postgresql
    - name: POSTGRES_USER
      value: postgres
    - name: POSTGRES_NAME
      value: sentry
    - name: POSTGRES_HOST
      value: sentry-sentry-postgresql
    - name: POSTGRES_PORT
      value: "5432"
    - name: SENTRY_ORG
      value: sentry

Here are the relevant logs from sentry-web:

07:47:39 [INFO] sentry.access.api: api.access (method='GET' view='sentry.web.frontend.auth_login.AuthLoginView' response=302 user_id='None' is_app='None' token_type='None' is_frontend_request='False' organization_id='None' auth_id='None' path='/auth/login/' caller_ip='-' user_agent='kube-probe/1.28' rate_limited='False' rate_limit_category='None' request_duration_seconds=0.0301055908203125 rate_limit_type='DNE' concurrent_limit='None' concurrent_requests='None' reset_time='None' group='None' limit='None' remaining='None')
07:47:39 [INFO] sentry.services.hybrid_cloud: Organization by slug [None] not found
06:51:04 [ERROR] sentry_sdk.errors: Unexpected status code: 502 (body: b'<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h

I think occured error in this fetch api call's code.

async function fetchOrg(
  api: Client,
  slug: string,
  usePreload?: boolean
): Promise<Organization> {
  const [org] = await getPreloadedDataPromise(
    'organization',
    slug,
    () =>
      // This data should get preloaded in static/sentry/index.ejs
      // If this url changes make sure to update the preload
      api.requestPromise(`/organizations/${slug}/`, {
        includeAllArgs: true,
        query: {detailed: 0, include_feature_flags: 1},
      }),
    usePreload
  );

  if (!org) {
    throw new Error('retrieved organization is falsey');
  }

( ... )
}

Could you provide guidance on resolving this issue?

tyr1k commented 1 month ago

did you run the sentry upgrade command inside the sentry-web pod?

choi2601 commented 1 month ago

@tyr1k Yes, I successfully ran sentry upgrade to migrate the database data, and both the user and organization were created in Sentry. However, the error persists.

tyr1k commented 1 month ago

@choi2601 what proxy do you use? show logs

choi2601 commented 1 month ago

@tyr1k hi, currently i am using Nginx as the proxy server.

Here are the logs:

Defaulted container "nginx" out of: nginx, preserve-logs-symlinks (init)
2024/08/21 05:36:59 [error] 48#48: *238880 connect() failed (111: Connection refused) while connecting to upstream, client: [REDACTED_IP], server: , request: "POST /api/1/envelope/ HTTP/1.1", upstream: "http://[REDACTED_IP]:3000/api/1/envelope/", host: "[REDACTED_HOST]"
[REDACTED_IP] - - [21/Aug/2024:05:37:00 +0000] "POST /api/1/envelope/ HTTP/1.1" 502 150 "-" "sentry.python/2.2.1" "-"
2024/08/21 05:37:00 [error] 48#48: *238882 connect() failed (111: Connection refused) while connecting to upstream, client: [REDACTED_IP], server: , request: "POST /api/1/envelope/ HTTP/1.1", upstream: "http://[REDACTED_IP]:3000/api/1/envelope/", host: "[REDACTED_HOST]"
2024/08/21 05:37:01 [error] 48#48: *238884 connect() failed (111: Connection refused) while connecting to upstream, client: [REDACTED_IP], server: , request: "POST /api/1/envelope/ HTTP/1.1", upstream: "http://[REDACTED_IP]:3000/api/1/envelope/", host: "[REDACTED_HOST]"
[REDACTED_IP] - - [21/Aug/2024:05:37:01 +0000] "POST /api/1/envelope/ HTTP/1.1" 502 150 "-" "sentry.python/2.2.1" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:04 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:04 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:04 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:09 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:09 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:09 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:14 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:14 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:14 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:19 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:19 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:19 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:24 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:24 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:24 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:29 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:29 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:29 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:34 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:34 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:34 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:39 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:39 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:39 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:44 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:44 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:44 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:49 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:49 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:49 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:54 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:54 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:54 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:59 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:59 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:59 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:37:59 +0000] "POST /api/1/envelope/ HTTP/1.1" 502 150 "-" "sentry.python/2.2.1" "-"
2024/08/21 05:37:59 [error] 48#48: *238880 connect() failed (111: Connection refused) while connecting to upstream, client: [REDACTED_IP], server: , request: "POST /api/1/envelope/ HTTP/1.1", upstream: "http://[REDACTED_IP]:3000/api/1/envelope/", host: "[REDACTED_HOST]"
[REDACTED_IP] - - [21/Aug/2024:05:38:00 +0000] "POST /api/1/envelope/ HTTP/1.1" 502 150 "-" "sentry.python/2.2.1" "-"
2024/08/21 05:38:00 [error] 48#48: *238882 connect() failed (111: Connection refused) while connecting to upstream, client: [REDACTED_IP], server: , request: "POST /api/1/envelope/ HTTP/1.1", upstream: "http://[REDACTED_IP]:3000/api/1/envelope/", host: "[REDACTED_HOST]"
2024/08/21 05:38:01 [error] 48#48: *238884 connect() failed (111: Connection refused) while connecting to upstream, client: [REDACTED_IP], server: , request: "POST /api/1/envelope/ HTTP/1.1", upstream: "http://[REDACTED_IP]:3000/api/1/envelope/", host: "[REDACTED_HOST]"
[REDACTED_IP] - - [21/Aug/2024:05:38:01 +0000] "POST /api/1/envelope/ HTTP/1.1" 502 150 "-" "sentry.python/2.2.1" "-"
[REDACTED_IP] - - [21/Aug/2024:05:38:04 +0000] "GET / HTTP/1.1" 302 0 "-" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:38:04 +0000] "GET /auth/login/ HTTP/1.1" 302 0 "http://[REDACTED_IP]:8080/" "kube-probe/1.28" "-"
[REDACTED_IP] - - [21/Aug/2024:05:38:04 +0000] "GET /auth/login/sentry/ HTTP/1.1" 200 17134 "http://[REDACTED_IP]:8080/auth/login/" "kube-probe/1.28" "-"

and nginx configuration is that:

nginx:
  enabled: true
  containerPort: 8080
  existingServerBlockConfigmap: '{{ template "sentry.fullname" . }}'
  resources: {}
  replicaCount: 1
  service:
    type: NodePort
    ports:
      http: 80
    nodePorts:
      http: "32510"
  extraLocationSnippet: false
  Access-Control-Allow-Origin: "*"
serrrios commented 1 month ago

Check routing url, / must be route to web, /api/* to relay.

choi2601 commented 1 month ago

Hi @serrrios, Thank you for your advice. I followed your instructions and updated the extraLocationSnippet in my values.yaml as follows:

  extraLocationSnippet: |
    location / {
      proxy_pass http://sentry-web:9000;
    }
    location /api/ {
      proxy_pass http://sentry-relay:3000;
    }

But persisent error occured. (Actually, I'm not entirely sure if I applied your advice correctly.)

What’s confusing me right now is how the /api/* routing is working. Some requests are receiving responses correctly, while others are resulting in errors.

Requests like the ones below are successed:

Request URL: http://[REDACTED_IP]/api/0/organizations/?member=1 Request Method: GET Status Code: 200 OK Remote Address: [REDACTED_IP]:32510 Referrer Policy: strict-origin-when-cross-origin

[
    {
        "id": "1",
        "slug": "sentry",
        "status": {
            "id": "active",
            "name": "active"
        },
        "name": "Sentry",
        "dateCreated": "2024-08-22T00:20:03.964241Z",
        "isEarlyAdopter": false,
        "require2FA": false,
        "requireEmailVerification": false,
        "avatar": {
            "avatarType": "letter_avatar",
            "avatarUuid": null,
            "avatarUrl": null
        },
        "links": {
            "organizationUrl": "http://[REDACTED_IP]:32510",
            "regionUrl": "http://[REDACTED_IP]:32510"
        },
        "hasAuthProvider": false,
        "features": [
            "session-replay-sdk",
            "relay",
            "sso-basic",
            "session-replay",
            "integrations-ticket-rules",
            "reprocessing-v2",
            "advanced-search",
            "discover-query",
            "integrations-chat-unfurl",
            "integrations-stacktrace-link",
            "large-debug-files",
            "session-replay-slack-new-issue",
            "transaction-metrics-extraction",
            "dashboards-edit",
            "integrations-alert-rule",
            "invite-members",
            "data-forwarding",
            "integrations-issue-sync",
            "issue-platform",
            "set-grouping-config",
            "related-events",
            "session-replay-recording-scrubbing",
            "integrations-issue-basic",
            "global-views",
            "shared-issues",
            "open-membership",
            "dashboards-rh-widget",
            "session-replay-sdk-errors-only",
            "symbol-sources",
            "alert-filters",
            "sso-saml2",
            "session-replay-ui",
            "discover-basic",
            "dashboards-basic",
            "integrations-incident-management",
            "session-replay-issue-emails",
            "integrations-event-hooks",
            "session-replay-new-event-counts",
            "event-attachments",
            "incidents",
            "onboarding",
            "invite-members-rate-limits",
            "dashboards-mep",
            "session-replay-a11y-tab",
            "performance-view",
            "discover",
            "custom-symbol-sources",
            "api-keys"
        ]
    }
]

Request URL: http://[REDACTED_IP]:32510/api/0/assistant/ Request Method: GET Status Code: 200 OK Remote Address: [REDACTED_IP]:32510 Referrer Policy: strict-origin-when-cross-origin

[
    {
        "guide": "issue",
        "seen": false
    },
    {
        "guide": "issue_stream",
        "seen": false
    },
    {
        "guide": "alerts_write_member",
        "seen": false
    },
    {
        "guide": "alerts_write_owner",
        "seen": false
    },
    {
        "guide": "trace_view",
        "seen": false
    },
    {
        "guide": "span_op_breakdowns_and_tag_explorer",
        "seen": false
    },
    {
        "guide": "team_key_transactions",
        "seen": false
    },
    {
        "guide": "project_transaction_threshold",
        "seen": false
    },
    {
        "guide": "project_transaction_threshold_override",
        "seen": false
    },
    {
        "guide": "semver",
        "seen": false
    },
    {
        "guide": "release_stages",
        "seen": false
    },
    {
        "guide": "new_page_filters",
        "seen": false
    },
    {
        "guide": "new_page_filters_pin",
        "seen": false
    },
    {
        "guide": "releases_widget",
        "seen": false
    },
    {
        "guide": "activate_sampling_rule",
        "seen": false
    },
    {
        "guide": "create_conditional_rule",
        "seen": false
    },
    {
        "guide": "explain_archive_button_issue_details",
        "seen": false
    },
    {
        "guide": "explain_archive_tab_issue_stream",
        "seen": false
    },
    {
        "guide": "explain_new_default_event_issue_detail",
        "seen": false
    },
    {
        "guide": "new_project_filter",
        "seen": false
    },
    {
        "guide": "explain_archive_button_issue_stream",
        "seen": false
    },
    {
        "guide": "ddm_view",
        "seen": false
    },
    {
        "guide": "metrics_onboarding",
        "seen": false
    }
]

Request URL: http://[REDACTED_IP]:32510/api/0/organizations/sentry/?detailed=0&include_feature_flags=1 Request Method: GET Status Code: 200 OK Remote Address: [REDACTED_IP]:32510 Referrer Policy: strict-origin-when-cross-origin

no respose

But when send a request to http://[REDACTED_IP]:32510/api/1/envelope/?sentry_key=[SENTRY_KEY]&sentry_version=7&sentry_client=sentry.javascript.react%2F8.0.0, receive

Request URL: http://[REDACTED_IP]:32510/api/1/envelope/?sentry_key=[SENTRY_KEY]&sentry_version=7&sentry_client=sentry.javascript.react%2F8.0.0 Request Method: POST Status Code: 502 Bad Gateway Remote Address: [REDACTED_IP]:32510 Referrer Policy: origin

Could you perhaps offer me some advice on how to deal with this terrible situation I'm in?

serrrios commented 1 month ago

It shouldn't have been taken so literally =) Look at how the Helm chart generates the config for Nginx or the official config for Self. https://github.com/getsentry/self-hosted/blob/master/nginx/nginx.conf After look logs on relay, web and wokers.

choi2601 commented 1 month ago

@serrrios Thank you for your answer, I'll try to search for relevant information :-)

archeoss commented 3 weeks ago

I had the similar-ish problem, check what the sentry-snuba-api pod logs are and if there is any errors, run snuba bootstrap --force on that pod. That fixed the problem for me

Although the problem origin can be entirely different

choi2601 commented 3 weeks ago

I was still struggling with that problem.​ I'll give it a try! Thank you​​​​​​​​​​​​​​​​🌝

zhouqunyi commented 1 week ago

check your disk space. try clear or remove docker logs if high disk usage exists.