The CLI does not work in a Github Codespace

[ouvreboite]

Describe the bug Hello Supabase devs :)

I'm trying to use the Supabase CLI inside a Github Codespace (to follow this guide), and I'm failing at various step.

I'm not very familiar with the Unix secret management, so I've tried to unblock myself and document my steps.

To Reproduce Create a codespace using the main image mcr.microsoft.com/devcontainers/typescript-node:1-20-bullseye and ghcr.io/devcontainers/features/docker-in-docker:2 to enable docker within the Codespace. I've defined two env variables:

• SUPABASE_ACCESS_TOKEN
• SUPABASE_DB_PASSWORD

1. Supabase init works well

   npx supabase init 
   🆗

   I just need to manually install Deno (curl -fsSL https://deno.land/install.sh at the start of my container) for the Deno VSCode plugin to work.

2. Supabase link fails (partially?)

   npx supabase link --project-ref my-project-ref  
   🟥Failed to save database password: failed to set credentials: exec: "dbus-launch": executable file not found in $PATH
   Finished supabase link.

   Installing dbus-x11

   sudo apt update
   sudo apt install dbus-x11 -y

   Retrying

   npx supabase link --project-ref my-project-ref 
   🟥 Failed to save database password: failed to set credentials: The name org.freedesktop.secrets was not provided by any .service files
   Finished supabase link.

   Installing keyring and libsecret

   sudo apt install gnome-keyring libsecret-1-0 -y

   Retrying

   npx supabase link --project-ref my-project-ref
   🟥 Failed to save database password: failed to set credentials: failed to unlock correct collection '/org/freedesktop/secrets/aliases/default'
   Finished supabase link.

Sudo (with -E to use my already defined env variables)

sudo -E npx supabase link --project-ref my-project-ref
^C(hangs forever so I manually kill it)
🟥 Failed to save database password: failed to set credentials: Failed to activate service 'org.freedesktop.secrets': timed out (service_start_timeout=120000ms)
Finished supabase link.

❓ I see that some work has been done to disable keyring for WSL, and I suppose we are in a similar situation (Codespace is essentially headless, except for the VSCode frontend). But the solution for WSL means that the token will be written to a file, which would be a problem for a Codespace (as we don't want passwords/tokens to be written in files, as they are stored in git)

3. Supabase DB pull fails

   npx supabase db pull --debug
   Using connection pooler: postgres://postgres.my-project-ref:@aws-0-eu-central-1.pooler.supabase.com:6543/postgres
   Supabase CLI 1.145.4
   2024/03/04 10:29:06 Sent Header: Host [api.moby.localhost]
   2024/03/04 10:29:06 Sent Header: User-Agent [Docker-Client/unknown-version (linux)]
   2024/03/04 10:29:06 Sent Header: X-Meta-Source-Client [github/codespaces]
   2024/03/04 10:29:06 Send Done
   2024/03/04 10:29:06 Recv First Byte
   Connecting to remote database...
   2024/03/04 10:29:06 Connect Start: udp 127.0.0.53:53
   2024/03/04 10:29:06 Connect Done: udp 127.0.0.53:53
   2024/03/04 10:29:06 Connect Start: udp 127.0.0.53:53
   2024/03/04 10:29:06 Connect Done: udp 127.0.0.53:53
   2024/03/04 10:29:06 Connect Start: udp 127.0.0.53:53
   2024/03/04 10:29:06 Connect Done: udp 127.0.0.53:53
   2024/03/04 10:29:06 Connect Start: udp 127.0.0.53:53
   2024/03/04 10:29:06 Connect Done: udp 127.0.0.53:53
   2024/03/04 10:29:06 Connect Start: tcp 18.198.145.223:6543
   2024/03/04 10:29:06 Connect Done: tcp 18.198.145.223:6543
   2024/03/04 10:29:06 PG Send: {"Type":"StartupMessage","ProtocolVersion":196608,"Parameters":{"database":"postgres","user":"postgres.my-project-ref"}}
   2024/03/04 10:29:06 PG Recv: {"Type":"AuthenticationSASL","AuthMechanisms":["SCRAM-SHA-256"]}
   [...] Some auth logs removed to avoid leaking tokens
   2024/03/04 10:29:06 PG Recv: {"Type":"AuthenticationOK"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"DateStyle","Value":"ISO, MDY"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"IntervalStyle","Value":"postgres"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"TimeZone","Value":"UTC"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"application_name","Value":"Supavisor"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"client_encoding","Value":"UTF8"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"default_transaction_read_only","Value":"off"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"in_hot_standby","Value":"off"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"integer_datetimes","Value":"on"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"is_superuser","Value":"off"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"server_encoding","Value":"UTF8"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"server_version","Value":"15.1 (Ubuntu 15.1-1.pgdg20.04+1)"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"session_authorization","Value":"postgres"}
   2024/03/04 10:29:06 PG Recv: {"Type":"ParameterStatus","Name":"standard_conforming_strings","Value":"on"}
   2024/03/04 10:29:06 PG Recv: {"Type":"BackendKeyData","ProcessID":117346820,"SecretKey":2730415841}
   2024/03/04 10:29:06 PG Recv: {"Type":"ReadyForQuery","TxStatus":"I"}
   2024/03/04 10:29:07 PG Send: {"Type":"Query","String":"SELECT version FROM supabase_migrations.schema_migrations ORDER BY version"}
   2024/03/04 10:29:07 PG Recv: {"Type":"RowDescription","Fields":[{"Name":"version","TableOID":28683,"TableAttributeNumber":1,"DataTypeOID":25,"DataTypeSize":-1,"TypeModifier":-1,"Format":0}]}
   2024/03/04 10:29:07 PG Recv: {"Type":"CommandComplete","CommandTag":"SELECT 0"}
   2024/03/04 10:29:07 PG Recv: {"Type":"ReadyForQuery","TxStatus":"I"}
   🟥 failed to open dump file: open supabase/migrations/20240304102906_remote_schema.sql: no such file or directory

   ❓ Is this related to the link secrets failure? Or something else? If I manually run create the migration folder mkdir supabase/migrations, the db pull succeeds

npx supabase db pull
Connecting to remote database...
(lots of pulls)
Status: Downloaded newer image for public.ecr.aws/supabase/postgres:15.1.0.147
Schema written to supabase/migrations/20240304122749_remote_schema.sql
Update remote migration history table? [Y/n] y
Repaired migration history: [20240304122749] => applied
Finished supabase db pull.
🆗Run supabase migration list to show the updated migration history.

❓Does it means the secret setting in the link operation is not required?

Expected behavior The Supabase CLI should work within a Github Codespace

Screenshots

System information Gihub Codespace

• main image: mcr.microsoft.com/devcontainers/typescript-node:1-20-bullseye
• feature image: ghcr.io/devcontainers/features/docker-in-docker:2 to enable docker within the Codespace

[ouvreboite]

Instead of a bug fix, maybe a guide/recipee on how to use Supabase within Github Codespace would be enough.

Currently, a setup that seems to work (I will update once I finished following the user management tutorial).

1. Create and assign the SUPABASE_ACCESS_TOKEN and SUPABASE_DB_PASSWORD codespace secrets to your repository
2. Create a codespace using a npm enabled base image (mcr.microsoft.com/devcontainers/typescript-node:1-20-bullseye for example)
   • manually run npx supabase init to create the supabase folder and config)
   • mkdir supabase/migrations
3. Update devcontainer.json to:
   • add the deno VSCode plugin (no need to keep the .vscode/extensions.json file)
   • add the docker-in-docker feature (ghcr.io/devcontainers/features/docker-in-docker:2)
   • add the deno executable each time the container starts (curl -fsSL https://deno.land/install.sh)
   • link to the database each time the container starts (npx --yes supabase link --project-ref your-project-id)
4. Running npx supabase db pull should work fine now

Example of devcontainer.json:

{
    "name": "Node.js & TypeScript",
    "image": "mcr.microsoft.com/devcontainers/typescript-node:1-20-bullseye",
    "customizations": {
        "vscode": {
            "extensions": [
                "denoland.vscode-deno"
            ]
        }
    },
    "postCreateCommand": "curl -fsSL https://deno.land/install.sh | sh && npx --yes supabase link --project-ref your-project-id",
    "features": {
        "ghcr.io/devcontainers/features/docker-in-docker:2": {}
    }
}

[rlataguerra]

the exact same problem happens when you use the cli inside a github action

[sweatybridge]

Thanks for the detailed report. I believe no. 2 and 3 can be fixed in code with better error handling.

I will ping our docs lead to check if we want to host the Codespaces setup docs on supabase domain.

[ouvreboite]

I've pushed my Codespaces/Supabase/SvelteKit experiment a bit farther, following this Build a User Management App with SvelteKit tutorial. So I'll use this issue to log some info for future users.

First, we need to differentiate two scenarios:

• Codespace in the browser:
  • I was able to go through the entire tutorial using a "production" Supabase project ✅
  • Using a local supabase setup is more difficult. One big pain point was that each port is forwarded as a distinct URL (localhost:PORT is exposed as https://CODESPACENAME-PORT.app.github.dev). This trigger CORS issues in the auth flow, for example. Maybe a workaround can be found, but I stopped there.🚫
• Codespace through a local VSCode:
  • Using a "production" Supabase project works fine✅
  • Using a local supabase can also work, because the local VSCode will act as a localhost proxy (localhost:PORT in the Codespace is exposed as localhost:PORT on your own computer by your local VSCode). So no problem with CORS.✅

Still, I've had to tweak some stuff to make everything work.

1. Making redirect work with Vite default port: the base port for Vite (and so SvelteKit) is not 3000, but 5173. So the auth block in config.toml must be updated

   [auth]
   # ...
   site_url = "http://127.0.0.1:5173" # use Vite port
   additional_redirect_urls = ["http://127.0.0.1:5173"]  # use Vite port

2. Making the magic link auth work locally: the auth.email block in config.toml must be updated

   [auth.email]
   # ...
   enable_confirmations = true

3. Exposing the Vite app through VSCode proxy: by default, npm run dev expose the SvelteKit app as localhost. Strangely, the VSCode proxy does not work (at least for me) here. But exposing explicitly as 127.0.0.1 works. So I've updated my package.json

   "scripts": {
   "dev": "vite dev --host 127.0.0.1",

4. Making the delete_storage_object function work locally: the tutorial create a delete_storage_object function to remove the old avatars. One key problem for a local run, is the hardcoded project URL and service-role keys directly in the SQL function to access the storage API.

   -- function delete_storage_object
   declare
   project_url text := '<YOURPROJECTURL>';
   service_role_key text := '<YOURSERVICEROLEKEY>'; --  full access needed
   url text := project_url||'/storage/v1/object/'||bucket||'/'||object;

   In addition to the security risk, hard-coding the "production" values would prevent the function from working locally (url and key are different). So I'm using the Vault feature to use different values locally and in prod. In the local seed.sql file I'm adding the service-role key the CLI returns after starting. For the API URL, i'm using http://host.docker.internal:54321 (cf https://github.com/orgs/supabase/discussions/13789#discussioncomment-8727771)

   select vault.create_secret('eyJhbGciOiJIUzI[...]', 'service-role-key');
   select vault.create_secret('http://host.docker.internal:54321', 'storage-url-for-database');

   Then I fetch the secrets in the delete_storage_object function

   -- function delete_storage_object
   declare
   storage_url text;
   service_role_key text;
   url text;
   begin
   storage_url := (select decrypted_secret from vault.decrypted_secrets where name = 'storage-url-for-database');
   service_role_key := (select decrypted_secret from vault.decrypted_secrets where name = 'service-role-key');
   url := storage_url||'/storage/v1/object/'||bucket||'/'||object;

   I just have to also add the secrets with the production values on my production app to have the function work both on local and production.

I don't know if this usage of Vault is best-practices. But if it is, I would suggest updating the various "Build a User Management App" tutorials to use it.