Test failures inside docker container dont cause pipeline to fail

[jmackereth]

Hi,

First off, great module - thanks for the effort! We are running tests via a bash script running inside a docker container. The container is set up as a service in docker-compose via a test run script which also sets up an empty SQL DB where we run tests:

UID="$(id -u)" GID="$(id -g)" ENVIRONMENT="azure" docker-compose -f docker-compose.yml config > docker-stack.yml
docker-compose -f docker-stack.yml build --no-cache
docker-compose -f docker-stack.yml down -v --remove-orphans
docker-compose -f docker-stack.yml up -d
docker-compose -f docker-stack.yml exec -T backend python /app/app/scripts/insert_scripts/create_database.py
docker-compose -f docker-stack.yml exec -T backend bash /app/app/scripts/test.sh "$@"
docker-compose -f docker-stack.yml down -v --remove-orphans
exit $?

The test.sh script itself is very simple:

pytest "$@" --cov /app/app --cov-report html 
exit $?

(I added the exit $? on the recommendation of an answer on this stack overflow post)

I noticed that the build doesnt fail if the tests fail. I would normally rectify this by setting the option failTaskOnFailedTests for the PublishTestResults@2 task, but this is abstracted away when using pytest-azurepipelines. How would I go about ensuring that the pipeline exits with a failure if any tests fail? Any help would be greatly appreciated!

just in case im doing something dumb, here is the pipeline.yml file:

trigger: none
pr:
  branches:
    include:
      - main

variables:
  - name: ENVIRONMENT
    value: "azure"
  - group: sor-data-api-ci-variables

pool:
  vmImage: "ubuntu-latest"

stages:
  - stage: test
    displayName: Build and Run Tests
    jobs:
      - job: CI

        steps:
          - script: |
              sh ./scripts/test-azure.sh
              exit $?
            displayName: Build and Run Unit Tests
            env:
              SQL_SERVER: $(sql-server)
              DATABASE_NAME: $(sql-database-name)
              SQL_UID: $(sql-uid)
              SQL_PWD: $(sql-pwd)
              SECRET_KEY: $(secret-key)
              OPENAPI_CLIENT_ID: $(openapi-client-id)
              APP_CLIENT_ID: $(app-client-id)
              TENANT_ID: $(tenant-id)
              REQUEST_SCOPE: $(request-scope)
              CONNECTION_STRING: "mssql+pyodbc:///?odbc_connect={}"
              DLS_ACCESS_STRING: "fake-string"