supabase / cli

Supabase CLI. Manage postgres migrations, run Supabase locally, deploy edge functions. Postgres backups. Generating types from your database schema.
https://supabase.com/docs/reference/cli/about
MIT License
1.09k stars 213 forks source link

Use a separate container for running pg_prove for db tests #1608

Closed vickkhera closed 11 months ago

vickkhera commented 1 year ago

Describe the bug Running supabase test db dies while installing pgTAP from CPAN.

To Reproduce Steps to reproduce the behavior:

  1. create a trivial test script that just check is auth.users is defined (not sure this is even necessary)
  2. run npx supabase test db

The test script:

BEGIN;
SELECT plan(1); -- only one statement to run
SELECT has_column('auth', 'users', 'id', 'id should exist');
SELECT * FROM finish();
ROLLBACK;

Expected behavior The test should run and pass.

Screenshots

[Vicks-M1-Air]% npx supabase test db --debug
Supabase CLI 1.106.1
2023/10/24 17:19:59 Sent Header: Host [api.moby.localhost]
2023/10/24 17:19:59 Sent Header: User-Agent [Docker-Client/unknown-version (darwin)]
2023/10/24 17:19:59 Sent Header: Content-Length [2048]
2023/10/24 17:19:59 Sent Header: Content-Type [text/plain]
2023/10/24 17:19:59 Send Done
2023/10/24 17:19:59 Recv First Byte
2023/10/24 17:19:59 Sent Header: Host [api.moby.localhost]
2023/10/24 17:19:59 Sent Header: User-Agent [Docker-Client/unknown-version (darwin)]
2023/10/24 17:19:59 Sent Header: Content-Length [1225]
2023/10/24 17:19:59 Sent Header: Content-Type [application/json]
2023/10/24 17:19:59 Send Done
2023/10/24 17:19:59 Recv First Byte
/bin/bash: line 8:  7335 Done                    echo
      7336 Killed                  | cpan -T TAP::Parser::SourceHandler::pgTAP &> /dev/null
2023/10/24 17:20:07 Sent Header: Host [api.moby.localhost]
2023/10/24 17:20:07 Sent Header: User-Agent [Docker-Client/unknown-version (darwin)]
2023/10/24 17:20:07 Send Done
2023/10/24 17:20:07 Recv First Byte
error executing command

Desktop (please complete the following information):

Additional context

I'm not running Docker Desktop, just plain CLI docker using an Alpine linux VM with colima. Everything else works splendidly.

The tests used to run just fine. I updated my schema today and wanted to re-run tests and add more before pushing to staging and production, but I ran into this error. It has been a while since I ran the tests so I don't know what version was working before.

sweatybridge commented 1 year ago

To help us debug, could you run supabase services and report the output here?

vickkhera commented 1 year ago
[Vicks-M1-Air]% npx supabase test db
/bin/bash: line 8:  8637 Done                    echo
      8638 Killed                  | cpan -T TAP::Parser::SourceHandler::pgTAP &> /dev/null
error executing command
Try rerunning the command with --debug to troubleshoot the error.
[Vicks-M1-Air]% npx supabase services

        SERVICE IMAGE      │        LOCAL         │ LINKED
  ─────────────────────────┼──────────────────────┼─────────
    supabase/postgres      │ 15.1.0.117           │ -
    supabase/gotrue        │ v2.99.0              │ -
    postgrest/postgrest    │ v11.2.0              │ -
    supabase/realtime      │ v2.10.1              │ -
    supabase/storage-api   │ v0.40.4              │ -
    supabase/edge-runtime  │ v1.22.3              │ -
    supabase/studio        │ 20231023-7e2cd92     │ -
    supabase/postgres-meta │ v0.68.0              │ -
    supabase/logflare      │ 1.4.0                │ -
    bitnami/pgbouncer      │ 1.20.1-debian-11-r39 │ -
    darthsim/imgproxy      │ v3.8.0               │ -

[Vicks-M1-Air]%
vickkhera commented 1 year ago

On a hunch, I bumped my docker VM's RAM to 4GB and it succeeded. The default was 2GB.

[Vicks-M1-Air]% npx supabase db test
supabase/tests/supabase/auth-users.sql .. ok
All tests successful.
Files=1, Tests=1,  0 wallclock secs ( 0.01 usr  0.00 sys +  0.01 cusr  0.00 csys =  0.02 CPU)
Result: PASS

I was expecting that once the initial install completed I could lower the RAM back to 2GB for the docker VM, but after stopping/reconfiguring/starting when I ran the test it was killed again trying to install the cpan module. The mid-point of 3GB seems to be enough RAM so I will leave it at that for now (my laptop only has 8).

Why does the docker image need to reinstall the pgTAP modules every time we run db test?

sweatybridge commented 1 year ago

Why does the docker image need to reinstall the pgTAP modules every time we run db test?

This might have been a side effect of db reset which recreates the database container.

I will repurpose this issue to improve the pg_prove installation process. One possible approach is to use a separate pg_prove container for running tests.

vickkhera commented 1 year ago

I'm not doing a db reset between tests, just stopping and restarting the docker host vm.