search

supabase / cli

Supabase CLI. Manage postgres migrations, run Supabase locally, deploy edge functions. Postgres backups. Generating types from your database schema.
https://supabase.com/docs/reference/cli/about
MIT License
1.09k stars 212 forks source link

Unhealtthy services on an empty project initialization #1959

Closed ayman-makki closed 5 months ago

ayman-makki commented 9 months ago

Describe the bug Crash report ID : df9051ce9f494342b5981d91e1f81fe9 I used to use the supabase locally without any issues and after resetting the project it started throwing

service not healthy: [supabase_rest_suptest supabase_edge_runtime_suptest]

without any clear reasons. I tried purging all docker volumes and images, switching to supabase-beta, nothing works. The weirdest is that the corresponding docker logs do not show any signs of error :

supabase_rest_suptest container logs: 19/Feb/2024:03:23:41 +0000: Starting PostgREST 12.0.1... 19/Feb/2024:03:23:41 +0000: Attempting to connect to the database... 19/Feb/2024:03:23:41 +0000: Connection successful 19/Feb/2024:03:23:41 +0000: Admin server listening on port 3001 19/Feb/2024:03:23:41 +0000: Listening on port 3000 19/Feb/2024:03:23:41 +0000: Config reloaded 19/Feb/2024:03:23:41 +0000: Listening for notifications on the pgrst channel 19/Feb/2024:03:23:41 +0000: Schema cache loaded

supabase_edge_runtime_suptest container logs: Serving functions on http://127.0.0.1:54321/functions/v1/ Using supabase-edge-runtime-1.36.1 (compatible with Deno v1.40.3)

And even after a few minutes, containers are still not healthy (after running with --ignore-health-checks)

CONTAINER ID   IMAGE                                             COMMAND                  CREATED         STATUS                   PORTS                                                                  NAMES
0286d002a0e4   public.ecr.aws/supabase/studio:20240205-b145c86   "docker-entrypoint.s…"   3 minutes ago   Up 3 minutes (healthy)   0.0.0.0:54323->3000/tcp, :::54323->3000/tcp                            supabase_studio_suptest
7c0a17662185   public.ecr.aws/supabase/postgres-meta:v0.75.0     "docker-entrypoint.s…"   3 minutes ago   Up 3 minutes (healthy)   8080/tcp                                                               supabase_pg_meta_suptest
10a91ce5e270   public.ecr.aws/supabase/edge-runtime:v1.36.1      "sh -c 'mkdir -p /ho…"   3 minutes ago   Up 3 minutes             8081/tcp                                                               supabase_edge_runtime_suptest
4e563505d45c   public.ecr.aws/supabase/imgproxy:v3.8.0           "imgproxy"               3 minutes ago   Up 3 minutes (healthy)   8080/tcp                                                               storage_imgproxy_suptest
daac27f02c04   public.ecr.aws/supabase/storage-api:v0.46.4       "docker-entrypoint.s…"   3 minutes ago   Up 3 minutes (healthy)   5000/tcp                                                               supabase_storage_suptest
03593e947430   public.ecr.aws/supabase/postgrest:v12.0.1         "/bin/postgrest"         3 minutes ago   Up 3 minutes             3000/tcp                                                               supabase_rest_suptest
d2f5ac07c4ce   public.ecr.aws/supabase/realtime:v2.25.50         "/usr/bin/tini -s -g…"   3 minutes ago   Up 3 minutes (healthy)   4000/tcp                                                               realtime-dev.supabase_realtime_suptest
47dfc2c3890c   public.ecr.aws/supabase/inbucket:3.0.3            "/start-inbucket.sh …"   3 minutes ago   Up 3 minutes (healthy)   1100/tcp, 2500/tcp, 0.0.0.0:54324->9000/tcp, :::54324->9000/tcp        supabase_inbucket_suptest
5386aa2ea318   public.ecr.aws/supabase/gotrue:v2.132.3           "auth"                   3 minutes ago   Up 3 minutes (healthy)   9999/tcp                                                               supabase_auth_suptest
c27a1819c333   public.ecr.aws/supabase/kong:2.8.1                "sh -c 'cat <<'EOF' …"   3 minutes ago   Up 3 minutes (healthy)   8001/tcp, 8443-8444/tcp, 0.0.0.0:54321->8000/tcp, :::54321->8000/tcp   supabase_kong_suptest
34b35f0cc8d4   public.ecr.aws/supabase/postgres:15.1.0.147       "sh -c 'cat <<'EOF' …"   4 minutes ago   Up 4 minutes (healthy)   0.0.0.0:54322->5432/tcp, :::54322->5432/tcp                            supabase_db_suptest

To Reproduce Steps to reproduce the behavior:

  1. mkdir supabase-test && cd supabase-test
  2. supabase init
  3. supabase start

Expected behavior All services should start normally as they used to

System information Rerun the failing command with --create-ticket flag (preferred).

sweatybridge commented 9 months ago

And even after a few minutes, containers are still not healthy (after running with --ignore-health-checks)

The services look healthy to me based on your output above, ie.

10a91ce5e270   public.ecr.aws/supabase/edge-runtime:v1.36.1      "sh -c 'mkdir -p /ho…"   3 minutes ago   Up 3 minutes             8081/tcp                                                               supabase_edge_runtime_suptest
03593e947430   public.ecr.aws/supabase/postgrest:v12.0.1         "/bin/postgrest"         3 minutes ago   Up 3 minutes             3000/tcp                                                               supabase_rest_suptest

The reason there's no (healthy) text is because these containers don't have a http client so we can't use the native docker health check. Instead, we issue health check from cli code. You can make the same request using curl to verify that they are healthy, ie.

$ curl -I localhost:54321/rest-admin/v1/ready
HTTP/1.1 200 OK
Connection: keep-alive
Date: Mon, 19 Feb 2024 10:36:03 GMT
Server: postgrest/12.0.2
Access-Control-Allow-Origin: *
X-Kong-Upstream-Latency: 1
X-Kong-Proxy-Latency: 0
Via: kong/2.8.1

Could you try isolating the problem further by excluding edge runtime from starting? ie. supabase start -x edge-runtime

ayman-makki commented 9 months ago

Thanks @sweatybridge for your inputs.

However, the check on postgrest is not producing expected output :

$ curl -I localhost:54321/rest-admin/v1/ready HTTP/1.1 502 Bad Gateway Date: Mon, 19 Feb 2024 16:28:15 GMT Content-Type: application/json; charset=utf-8 Connection: keep-alive Content-Length: 75 Access-Control-Allow-Origin: * X-Kong-Upstream-Latency: 0 X-Kong-Proxy-Latency: 0 Via: kong/2.8.1

Again, logs checking does not bring anything unusual.

Could you try isolating the problem further by excluding edge runtime from starting? ie. supabase start -x edge-runtime

service not healthy: [supabase_rest_institut]

Similaryl, testing supabase start -x postgrest service not healthy: [supabase_edge_runtime_institut]

paul-pro commented 9 months ago

FTR: I was having the same problem and googled this issue. After a while I've figured out that colima wasn't starting since my docker setup was broken (installed via app store and colima via brew so they weren't linked together - this wasn't obvious until I've checked brew services list and cat /opt/homebrew/var/log/colima.log).

So, removing docker app and reinstalling it from brew fixed colima and get supabase start to run w/o errors

sweatybridge commented 5 months ago

I've improved health check implementation a little. Let me know if this is still reproducible using npx supabase@beta start