grafana / oncall

Developer-friendly incident response with brilliant Slack integration
GNU Affero General Public License v3.0
3.52k stars 291 forks source link

helm: problem with connect to postgres #3052

Open batazor opened 1 year ago

batazor commented 1 year ago

What went wrong?

What happened:

What did you expect to happen:

How do we reproduce it?

  1. Install Helm chart: Grafana OnCall last version

Error:

with self.cursor() as cursor:
         ^^^^^^^^^^^^^  File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 33, in inner    return func(*args, **kwargs)           ^^^^^^^^^^^^^^^^^^^^^  
File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 259, in cursor    return self._cursor()           ^^^^^^^^^^^^^^  
File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 235, in _cursor    self.ensure_connection()  
File "/usr/local/lib/python3.11/site-packages/django_dbconn_retry/apps.py", line 69, in ensure_connection_with_retries    self.ensure_connection()  
File "/usr/local/lib/python3.11/site-packages/django_dbconn_retry/apps.py", line 48, in ensure_connection_with_retries    with self.wrap_database_errors:  
File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 90, in __exit__    raise dj_exc_value.with_traceback(traceback) from exc_value  
File "/usr/local/lib/python3.11/site-packages/django_dbconn_retry/apps.py", line 51, in ensure_connection_with_retries    self.connect()  File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 33, in inner    return func(*args, **kwargs)           ^^^^^^^^^^^^^^^^^^^^^  
File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 200, in connect    self.connection = self.get_new_connection(conn_params)                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 33, in inner    return func(*args, **kwargs)           ^^^^^^^^^^^^^^^^^^^^^  File "/usr/local/lib/python3.11/site-packages/django/db/backends/postgresql/base.py", line 187, in get_new_connection    connection = Database.connect(**conn_params)                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File "/usr/local/lib/python3.11/site-packages/psycopg2/__init__.py", line 122, in connect    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.OperationalError: connection to server at "oncall-postgres-ha.grafana-oncall" (10.233.5.189), port 5432 failed: Connection refused  Is the server running on that host and accepting TCP/IP connections?
Waiting for database migrations

Grafana OnCall Version

v1.3.38

Product Area

Helm

Grafana OnCall Platform?

Kubernetes

User's Browser?

No response

Anything else to add?

No response

batazor commented 1 year ago

I have tried both default Postgres from the chart and external and crunchdata-postgres - in all cases, there is a connection problem

Asgoret commented 11 months ago

Same story. U found workaround?

Asgoret commented 11 months ago

Just wait for migrations...what migrations? Where?

batazor commented 11 months ago

@Asgoret see https://github.com/grafana/oncall/blob/dev/helm/oncall/values.yaml#L382-L389

Asgoret commented 11 months ago

@batazor don't get it...disable migrations?

batazor commented 11 months ago

@Asgoret The other services expect it to complete, even with migration turned off, so I don't think this is a workable option (I tried it, didn't work)