Test environment profiles

[colesturza]

Description

It would be nice to be able to set different testing profiles (e.g.: local, ci, etc.). Presently the only test profile is "test". This feature would allow separate application properties to be selected depending on where the tests run. For instance, my team uses test containers and dev services locally, but we use Gitlab CI services within our CI pipeline to deploy separate containers. Presently, we've created a Gradle task to copy over the appropriate properties to the build folder depending on if a Gradle project property is set. However, we'd like to be able to set application properties within src/test/resources/application.properties like the following.

quarkus.datasource.db-kind=postgresql

%local.quarkus.datasource.devservices.image-name=...

%ci.quarkus.datasource.devservices.enabled=false
%ci.quarkus.datasource.username=quarkus_ci_test
%ci.quarkus.datasource.password=quarkus_ci_test

%ci.quarkus.datasource.jdbc.url=jdbc:postgresql://postgres:5432/quarkus_ci_test

Implementation ideas

No response

[x80486]

On the other hand, you can take advantage on the fact that Quarkus starts in certain modes: dev, test, prod —rather than "profiles"— and make use of environment variables. It will simplify your workflow and will get you closer to a more truly twelve-factor application, but eventually far away from the bootified mindset.

quarkus:
  ...
  datasource:
    db-kind: postsgresql
    jdbc:
      url: ${DATASOURCE_JDBC_URL:?error}
    password: ${DATASOURCE_JDBC_PASSWORD:?error}
    username: ${DATASOURCE_JDBC_USERNAME:?error}
  flyway:
    clean-at-start: false
    clean-disabled: true
    locations: classpath:database/migrations/
    migrate-at-start: true
    validate-on-migrate: true
  ...
---
"%dev":
  quarkus:
    flyway:
      clean-at-start: true
      clean-disabled: false
---
"%prod":
---
"%test":
  quarkus:
    flyway:
      clean-at-start: true
      clean-disabled: false

As you can see, some properties are externalized, but some others not — it's a balance that will be up to you.

The multiple profiles route will deviate you at some point into aligning with environments and what not. In the end both approaches work similarly; one requires a lot of Jiu Jitsu, the other one is simpler, but will make you externalize the configuration more.

[colesturza]

@x80486 We do already make use of the dev, test, and prod modes. This issue mostly arose when trying to utilize dev services like test containers to spin up a database in a docker container, apposed to having to use docker-compose. We utilize GitLab CI for our pipelines, so we have the ability to run other containers adjacent to a test job (via GitLab services).

I found it a bit more confusing to others when the properties file ended up looking like the following:

# needs to be set to false for CI and true for local development
%test.quarkus.datasource.devservices.enabled=${DATASOURCE_DEVSERVICES_ENABLED}

# local dev env
%test.quarkus.datasource.devservices.image-name=${DATASOURCE_DEVSERVICES_IMAGE}
%test.quarkus.datasource.devservices.db-name=${DATASOURCE_DEVSERVICES_DB}
%test.quarkus.datasource.devservices.username=${DATASOURCE_DEVSERVICES_USER}
%test.quarkus.datasource.devservices.password=${DATASOURCE_DEVSERVICES_PASSWORD}

# ci env
%test.quarkus.datasource.username=${DATASOURCE_USER}
%test.quarkus.datasource.password=${DATASOURCE_PASSWORD}
%test.quarkus.datasource.jdbc.url=${DATASOURCE_JDBC_URL}

# common
%test.quarkus.datasource.db-kind=postgresql
%test.quarkus.flyway.migrate-at-start=true

We could totally switch to what you described with environment variables and self-managed containers through docker-compose. This would definitely alleviate the issue.

[x80486]

Based on what you mention, you have almost exactly the same setup that I have. I'm using Dev Services for everything-local — not using any custom environment variables anymore.

quarkus:
  datasource:
    db-kind: postgresql
    devservices:
      db-name: quarkus
      image-name: postgres:16.1-alpine
      properties:
        search_path: ${quarkus.liquibase.default-schema-name}

That configuration works for local development and testing. When I run the tests in GitLab CI pipeline I disable Dev Services (because the Testcontainers setup takes too much time) and use instead the GitLab CI services:

default:
  ...
  image: eclipse-temurin:21-jdk-jammy

...

variables:
  POSTGRES_DB: "quarkus"
  POSTGRES_PASSWORD: "$CI_COMMIT_SHA"
  POSTGRES_USER: "$CI_PROJECT_NAME"
  QUARKUS_DATASOURCE_JDBC_URL: "jdbc:postgresql://postgres:5432/quarkus"
  QUARKUS_DATASOURCE_PASSWORD: "$CI_COMMIT_SHA"
  QUARKUS_DATASOURCE_USERNAME: "$CI_PROJECT_NAME"
  QUARKUS_DEVSERVICES_ENABLED: "false"

test:
  script:
    - ./gradlew test
  services:
    - postgres:16.1-alpine
  stage: test

But I never mangle with the three default modes. If you have some deployment environments, you would need to provide only: QUARKUS_DATASOURCE_JDBC_URL, QUARKUS_DATASOURCE_PASSWORD, and QUARKUS_DATASOURCE_USERNAME — because there won't be default values for these settings in prod based on this setup.

It becomes a little bit more complex if you have more "services" (e.g.: Kafka, RabbitMQ, etc.) but I've managed to remove the Docker Compose file in Quarkus by using Dev Services.

[colesturza]

That looks like a good solution to me. Thanks for sharing.

[radcortez]

This is possible using multiple profiles or the parent profile: https://quarkus.io/guides/config-reference#multiple-profiles https://quarkus.io/guides/config-reference#parent-profile