Issue connecting to any data source

[iRakic]

Hello everyone,

I'm using the latest Redash chart (3.0.1). It was deployed using the following values:

ingress:  
  enabled: true  
  ingressClassName: "nginx"
  hosts:
    - host: redash.redacted-domain.tld
      paths:
        - "/"

redash:
  cookieSecret: REDACTED-FOR-SECURITY
  secretKey: REDACTED-FOR-SECURITY

postgresql:
  postgresqlPassword: REDACTED-FOR-SECURITY
redis:
  password: REDACTED-FOR-SECURITY

However, when I try to run test for datasource that I have added I get the following error: "Connection Test Failed: Unknown error occurred while performing connection test. Please try again later." The /api/data_sources/1/test POST request is responding with HTTP 502 and the execution time is >= 30s

Can you please tell me what is wrong or maybe point me to the right direction?

Thank you in advance!

[justinclift]

@AndrewChubatiuk Sounds like one for you? :smile:

[chenchuk77]

Hi, I got a similar issue. When testing a datasource, the UI timeouts after a while and this appears on logs :

[redash-7c94b854dc-8p2h8] [2024-04-22 07:16:49 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:9) [redash-7c94b854dc-8p2h8] [2024-04-22 07:16:49,196][PID:9][INFO][metrics] method=POST path=/api/data_sources/1/test endpoint=datasourcetestresource status=500 content_type=? content_length=-1 duration=60422.93 query_count=4 query_duration=7.50 [redash-7c94b854dc-8p2h8] [2024-04-22 07:16:49 +0000] [9] [ERROR] Error handling request /api/data_sources/1/test [redash-7c94b854dc-8p2h8] Traceback (most recent call last): [redash-7c94b854dc-8p2h8] File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 135, in handle [redash-7c94b854dc-8p2h8] self.handle_request(listener, req, client, addr) [redash-7c94b854dc-8p2h8] File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 178, in handle_request [redash-7c94b854dc-8p2h8] respiter = self.wsgi(environ, resp.start_response) [redash-7c94b854dc-8p2h8] File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2213, in call [redash-7c94b854dc-8p2h8] return self.wsgi_app(environ, start_response) [redash-7c94b854dc-8p2h8] File "/usr/local/lib/python3.8/site-packages/werkzeug/middleware/proxy_fix.py", line 182, in call [redash-7c94b854dc-8p2h8] return self.app(environ, start_response) [redash-7c94b854dc-8p2h8] File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2190, in wsgi_app [redash-7c94b854dc-8p2h8] response = self.full_dispatch_request() [redash-7c94b854dc-8p2h8] File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1484, in full_dispatch_request [redash-7c94b854dc-8p2h8] rv = self.dispatch_request() [redash-7c94b854dc-8p2h8] File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1469, in dispatch_request [redash-7c94b854dc-8p2h8] return self.ensure_sync(self.view_functions[rule.endpoint])(view_args) [redash-7c94b854dc-8p2h8] File "/usr/local/lib/python3.8/site-packages/flask_restful/init.py", line 489, in wrapper [redash-7c94b854dc-8p2h8] resp = resource(*args, *kwargs) [redash-7c94b854dc-8p2h8] File "/usr/local/lib/python3.8/site-packages/flask_login/utils.py", line 277, in decorated_view [redash-7c94b854dc-8p2h8] return current_app.ensure_sync(func)(args, kwargs) [redash-7c94b854dc-8p2h8] File "/usr/local/lib/python3.8/site-packages/flask/views.py", line 109, in view [redash-7c94b854dc-8p2h8] return current_app.ensure_sync(self.dispatch_request)(kwargs) [redash-7c94b854dc-8p2h8] File "/app/redash/handlers/base.py", line 31, in dispatch_request [redash-7c94b854dc-8p2h8] return super(BaseResource, self).dispatch_request(*args, *kwargs) [redash-7c94b854dc-8p2h8] File "/usr/local/lib/python3.8/site-packages/flask_restful/init.py", line 604, in dispatch_request [redash-7c94b854dc-8p2h8] resp = meth(args, kwargs) [redash-7c94b854dc-8p2h8] File "/app/redash/permissions.py", line 71, in decorated [redash-7c94b854dc-8p2h8] return fn(*args, **kwargs) [redash-7c94b854dc-8p2h8] File "/app/redash/handlers/data_sources.py", line 232, in post [redash-7c94b854dc-8p2h8] time.sleep(1) [redash-7c94b854dc-8p2h8] File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 203, in handle_abort [redash-7c94b854dc-8p2h8] sys.exit(1) [redash-7c94b854dc-8p2h8] SystemExit: 1 [redash-7c94b854dc-8p2h8] [2024-04-22 07:16:49 +0000] [9] [INFO] Worker exiting (pid: 9) [redash-7c94b854dc-8p2h8] [2024-04-22 07:16:50 +0000] [1] [ERROR] Worker (pid:9) was sent SIGKILL! Perhaps out of memory? [redash-7c94b854dc-8p2h8] [2024-04-22 07:16:50 +0000] [103] [INFO] Booting worker with pid: 103

[AndrewChubatiuk]

@iRakic are you using image provided with chart version 3.0.1 or do you set a custom one?

[iRakic]

@iRakic are you using image provided with chart version 3.0.1 or do you set a custom one?

I'm using the one from the chart.

[AndrewChubatiuk]

which datasource type are you trying to query? does it return lots of data during these attempts to query? have you tried to set server.resources?

[iRakic]

@AndrewChubatiuk I guess this last reply is for @chenchuk77 In my tests, I didn't encounter out of memory since server.resources is not set.

which datasource type are you trying to query? I have tried multiple databases, such as MSSQL, Postgres, and MariaDB, as a test.

I have also noticed this when I have tried to run a test under data source: [2024-04-22 16:21:55 +0000] [7] [CRITICAL] WORKER TIMEOUT (pid:2170) . It is from redash pod

[AndrewChubatiuk]

you may have resources not set but do you have enough free ram on machine?

[iRakic]

you may have resources not set but do you have enough free ram on machine?

Yes, like at least 64GB of RAM at all times. It's a cluster of multiple bare-metal servers.

[chenchuk77]

Hi @AndrewChubatiuk Here is my values :

  resources:
    limits:
      cpu: 500m
      memory: 3Gi
    requests:
      cpu: 500m
      memory: 2Gi

I tried a datasource of type postgres (which is accessible from the pod shell). i also try JSON from a url. both timedout and i see similiar output in logs