search

supabase / cli

Supabase CLI. Manage postgres migrations, run Supabase locally, deploy edge functions. Postgres backups. Generating types from your database schema.
https://supabase.com/docs/reference/cli/about
MIT License
1.07k stars 209 forks source link

Update vector to >=0.3.0 to support remote docker hosts #2531

Closed jweberde closed 3 months ago

jweberde commented 3 months ago

Bug report

Describe the bug

Running the Supabase CLI Version 1.183.5 clean installation using docker with a remote docker endpoint e.g. tcp://192.168.10.10:2375 will lead to startup error of the vector container. While starting the project using npx supabase start the vector container will timeout with health check failure and therefore the whole start is canceled:

supabase_vector_project_id container is not ready: unhealthy

Detailed Docker Logs

docker logs 191a972a7de0
2024-07-17T07:54:53.803819Z  INFO vector::app: Internal log rate limit configured. internal_log_rate_secs=10
2024-07-17T07:54:53.803939Z  INFO vector::app: Log level is enabled. level="vector=info,codec=info,vrl=info,file_source=info,tower_limit=trace,rdkafka=info,buffers=info,lapin=info,kube=info"
2024-07-17T07:54:53.803995Z  INFO vector::app: Loading configs. paths=["/etc/vector/vector.yaml"]
2024-07-17T07:54:53.829575Z  WARN vector::config::loading: Transform "router._unmatched" has no consumers
2024-07-17T07:54:54.075426Z ERROR vector::topology: Configuration error. error=Source "docker_host": Unknown scheme: tcp
2024-07-17T07:54:54.634091Z  INFO vector::app: Internal log rate limit configured. internal_log_rate_secs=10
2024-07-17T07:54:54.634222Z  INFO vector::app: Log level is enabled. level="vector=info,codec=info,vrl=info,file_source=info,tower_limit=trace,rdkafka=info,buffers=info,lapin=info,kube=info"
2024-07-17T07:54:54.634285Z  INFO vector::app: Loading configs. paths=["/etc/vector/vector.yaml"]
2024-07-17T07:54:54.663345Z  WARN vector::config::loading: Transform "router._unmatched" has no consumers
2024-07-17T07:54:54.863284Z ERROR vector::topology: Configuration error. error=Source "docker_host": Unknown scheme: tcp

Seems to be an error with the Remote TCP Docker Host

Configuration error. error=Source "docker_host": Unknown scheme: tcp

To Reproduce

Configure your Mac Local Docker Daemon to be exposed via TCP. For example via socat

  1. Setup Docker Daemon via TCP not unix socket. 
    
docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 127.0.0.1:1234:1234 bobrik/socat TCP-LISTEN:1234,fork UNIX-CONNECT:/var/run/docker.sock

expose as default docker host for the current running terminal session

export DOCKER_HOST=tcp://localhost:1234

verify access to docker daemon

docker ps -a

2.  Init Vanilla Supabase Project

```bash
npx supabase init
  1. Start Project
npx supabase start
  1. Monitor Terminal for Errors
supabase_vector_project_id container is not ready: unhealthy

Expected behavior

The supabase vector container should support Remote Docker Daemon

Screenshots

If applicable, add screenshots to help explain your problem.

System information

Workaround which will disable Vector for now

# Stop all existing containers deleting their volumes
npx supabase stop --no-backup

# Start without vector in case you are not using it
npx supabase start -x vector
encima commented 3 months ago

Thanks for reporting this @jweberde (and for the workaround). We use an unmodified timberio/vector image in Supabase (currently timberio/vector:0.28.1-alpine) so the issue is with the image there.

Would you mind opening the issue with them as a feature request and linking to this issue? I will close this one as it is not something we can support without a custom build.

Thanks!

jweberde commented 3 months ago

@encima I discussed the topic with vector community and they told me that in Vector >= timberio/vector:0.30 (May) the issue has been fixed and tcp scheme is supported.

I was able to find the fix - https://github.com/vectordotdev/vector/commit/e1c0c0275a32ac70a6371ece23f8283abe19880c and verify the comment given by the community. https://vector.dev/releases/0.30.0/

So basically the problem is because of an outdated version of vector used by supabase. Unfortunately I am not able to overwrite the vector image version myself as it is not mapped: https://github.com/supabase/cli/blob/develop/pkg/config/config.go#L361

encima commented 3 months ago

Thanks for checking this @jweberde !

OK, let's rename the issue to request the image to be updated in the latest CLI release.

As a workaround, you can use the Docker self hosting method and specify the image version in the docker-compose.yml

sweatybridge commented 3 months ago

I've merged a fix in cli 1.187.5 so it sends the http scheme to vector instead of tcp. Generally our local and self-service versions match hosted for reproducibility so upgrading vector to a new version may not be trivial.

But I will let @Ziinc decide on that.

arikanf commented 2 months ago

This problem still exists even though upgrading vector to 0.30.0 on macos sonoma with the latest docker desktop

✔ Network supabase_default Created 0.0s ✔ Container supabase-imgproxy Created 0.1s ✔ Container supabase-vector Created 0.1s ✔ Container supabase-db Created 0.0s ✔ Container supabase-analytics Created 0.0s ✔ Container supabase-kong Created 0.0s ✔ Container supabase-studio Created 0.0s ✔ Container realtime-dev.supabase-realtime Created 0.0s ✔ Container supabase-meta Created 0.0s ✔ Container supabase-rest Created 0.0s ✔ Container supabase-auth Created 0.0s ✔ Container supabase-edge-functions Created 0.0s ✔ Container supabase-storage Created 0.0s Attaching to realtime-dev.supabase-realtime, supabase-analytics, supabase-auth, supabase-db, supabase-edge-functions, supabase-imgproxy, supabase-kong, supabase-meta, supabase-rest, supabase-storage, supabase-studio, supabase-vector supabase-imgproxy | WARNING [2024-08-14T08:41:13Z] No keys defined, so signature checking is disabled supabase-imgproxy | WARNING [2024-08-14T08:41:13Z] No salts defined, so signature checking is disabled supabase-imgproxy | WARNING [2024-08-14T08:41:13Z] Exposing root via IMGPROXY_LOCAL_FILESYSTEM_ROOT is unsafe supabase-vector | 2024-08-14T08:41:13.849447Z INFO vector::app: Log level is enabled. level="vector=info,codec=info,vrl=info,file_source=info,tower_limit=trace,rdkafka=info,buffers=info,lapin=info,kube=info" supabase-vector | 2024-08-14T08:41:13.851080Z INFO vector::app: Loading configs. paths=["etc/vector/vector.yml"] supabase-imgproxy | INFO [2024-08-14T08:41:13Z] Starting server at :5001 supabase-vector | 2024-08-14T08:41:13.865626Z WARN vector::config::loading: Transform "router._unmatched" has no consumers supabase-vector | 2024-08-14T08:41:13.865993Z INFO source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::sources::docker_logs: Capturing logs from now on. now=2024-08-14T08:41:13.865927844+00:00 supabase-vector | 2024-08-14T08:41:13.866064Z INFO source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::sources::docker_logs: Listening to docker log events. supabase-vector | 2024-08-14T08:41:13.951340Z INFO vector::topology::running: Running healthchecks. supabase-vector | 2024-08-14T08:41:13.951437Z INFO vector::topology::builder: Healthcheck passed. supabase-vector | 2024-08-14T08:41:13.951453Z INFO vector::topology::builder: Healthcheck passed. supabase-vector | 2024-08-14T08:41:13.951460Z INFO vector::topology::builder: Healthcheck passed. supabase-vector | 2024-08-14T08:41:13.951463Z INFO vector::topology::builder: Healthcheck passed. supabase-vector | 2024-08-14T08:41:13.951465Z INFO vector::topology::builder: Healthcheck passed. supabase-vector | 2024-08-14T08:41:13.951467Z INFO vector::topology::builder: Healthcheck passed. supabase-vector | 2024-08-14T08:41:13.951490Z INFO vector::topology::builder: Healthcheck passed. supabase-vector | 2024-08-14T08:41:13.951557Z INFO vector: Vector has started. debug="false" version="0.30.0" arch="aarch64" revision="38c3f0b 2023-05-22 17:38:48.655488673" supabase-vector | 2024-08-14T08:41:13.951723Z ERROR source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::sources::docker_logs: Listing currently running containers failed. error=error trying to connect: Connection refused (os error 111) supabase-vector | 2024-08-14T08:41:13.954824Z INFO vector::internal_events::api: API server running. address=0.0.0.0:9001 playground=http://0.0.0.0:9001/playground supabase-vector | 2024-08-14T08:41:13.954866Z INFO vector_common::shutdown: All sources have finished. supabase-vector | 2024-08-14T08:41:13.954868Z INFO vector_common::shutdown: All sources have finished. supabase-vector | 2024-08-14T08:41:13.954880Z INFO vector::app: All sources have finished. supabase-vector | 2024-08-14T08:41:13.954888Z INFO vector: Vector has stopped. supabase-vector | 2024-08-14T08:41:13.956640Z INFO vector::topology::running: Shutting down... Waiting on running components. remaining_components="logflare_db, logflare_auth, logflare_storage, logflare_kong, logflare_realtime, logflare_rest, logflare_functions" time_remaining="59 seconds left" supabase-vector exited with code 0 Gracefully stopping... (press Ctrl+C again to force) dependency failed to start: container supabase-vector exited (0)

2024-08-14T08:41:13.951557Z INFO vector: Vector has started. debug="false" version="0.30.0" arch="aarch64" revision="38c3f0b 2023-05-22 17:38:48.655488673" 2024-08-14T08:41:13.951723Z ERROR source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::sources::docker_logs: Listing currently running containers failed. error=error trying to connect: Connection refused (os error 111) 2024-08-14T08:41:13.954824Z INFO vector::internal_events::api: API server running. address=0.0.0.0:9001 playground=http://0.0.0.0:9001/playground 2024-08-14T08:41:13.954866Z INFO vector_common::shutdown: All sources have finished. 2024-08-14T08:41:13.954868Z INFO vector_common::shutdown: All sources have finished. 2024-08-14T08:41:13.954880Z INFO vector::app: All sources have finished. 2024-08-14T08:41:13.954888Z INFO vector: Vector has stopped. 2024-08-14T08:41:13.956640Z INFO vector::topology::running: Shutting down... Waiting on running components. remaining_components="logflare_db, logflare_auth, logflare_storage, logflare_kong, logflare_realtime, logflare_rest, logflare_functions" time_remaining="59 seconds left"

sweatybridge commented 2 months ago

@arikanf are you using latest cli or self-hosting?

arikanf commented 2 months ago

self-hosting with docker

sweatybridge commented 2 months ago

Can you verify that /var/run/docker.sock is enabled under docker settings > advanced.

And also that docker socket is volume mounted into vector container? https://github.com/supabase/supabase/blob/master/docker/docker-compose.yml#423

arikanf commented 2 months ago
image image image image image image