buildkite-plugins / docker-compose-buildkite-plugin

🐳⚡️ Run build scripts, and build + push images, w/ Docker Compose
MIT License
172 stars 140 forks source link

Docker volume in database container not mounted when running docker-compose with database dependency #361

Closed wahyudibo closed 1 year ago

wahyudibo commented 1 year ago

Hi guys, i have a problem running an integration test using docker-compose consisting of web service and database. So, here's my pipeline.yml:

steps:
  - label: "Running test"
    agents:
      queue: "default"
    plugins:
      - docker-compose#v4.5.0:
          run: test
          config: docker-compose.yml

and here's my docker-compose.yml:

version: '3.8'
services:
  test:
    build:
      context: .
      target: builder
    command: ./scripts/test.sh
    depends_on:
      database:
        condition: service_healthy
    environment:
      - POSTGRES_HOST=database
      - POSTGRES_PORT=5432
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRES_DATABASE=${POSTGRES_DATABASE}

  database:
    image: postgres:13-alpine
    command: postgres -c 'max_connections=300'
    environment:
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRES_DB=${POSTGRES_DATABASE}
    volumes:
      - "./data/postgres/:/docker-entrypoint-initdb.d/"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready"]
      interval: 10s
      timeout: 5s
      retries: 5

you can see in the database section, i have mounted volume that contains schema definition to create tables and fill it up with seed data. It running successfully in my local using docker-compose command but i get this error in the pipeline:

Building Docker Compose Service: test | 2s
  | Starting dependencies | 13s
  | $ docker-compose -f docker-compose.yml -p buildkite01847ec7f81c49b08dc5dec0856aede0 up -d --scale test=0 test
  | Creating network "buildkite01847ec7f81c49b08dc5dec0856aede0_default" with the default driver
  | Creating buildkite01847ec7f81c49b08dc5dec0856aede0_database_1 ... done
  |  
  | Running in service test | 24s
  | $ docker-compose -f docker-compose.yml -p buildkite01847ec7f81c49b08dc5dec0856aede0 run --name buildkite01847ec7f81c49b08dc5dec0856aede0_test_build_39 --rm test
  | Creating buildkite01847ec7f81c49b08dc5dec0856aede0_test_run ... done
  | ?       github.com/wahyudibo/try-buildkite-go-project/cmd/api   [no test files]
  | ?       github.com/wahyudibo/try-buildkite-go-project/internal/application/user/dao [no test files]
  | === RUN   TestGetUserByIDFeatureSuccess
  | respBody: {"error":"ERROR: relation \"users\" does not exist (SQLSTATE 42P01)"} <- THIS IS THE ERROR

The error indicates that the table users is not there yet. I try to put a sleep at the beginning of the test to ensure that this is not some data race problem but it seems not. My guess is somehow the volume in the database is not mounted in the pipeline so the database doesn't pick up the schema and run it at the database initialization. Please kindly suggests. Thank you

toote commented 1 year ago

@wahyudibo that is a very weird situation. If you add the upload-container-logs: always option to your step's plugin configuration you should be able to get the complete output of the postgresql logs that should allow us all continue to troubleshoot this issue.