Devops pipeline doesn't fail even though testing failed in cluster job.

[Halpph]

Hello, when I'm running some integration tests and I get an error I can see something like this on the cluster's logs ====================================================================== FAIL: test_datawarehouse (main.DatawarehouseIntegrationTest)

Traceback (most recent call last): File "", line 116, in test_datawarehouse File "", line 247, in test_mergePk AssertionError

---

Ran 2 tests in 210.850s

FAILED (failures=1)

Then I'm expecting to see a failure in the related Azure devops pipeline but that's not the case, how could I make the pipeline fail if there's an error in the job?

On azure devops i see the following: [dbx][2021-01-11 15:17:10.918] Job run is not yet finished, current status message: In run [dbx][2021-01-11 15:17:16.011] Job run finished successfully [dbx][2021-01-11 15:17:16.011] Launch command finished

[renardeinside]

Hi @Halpph !

Could you please tell me - do you pass a --trace parameter in the CI pipeline?

[Halpph]

Yes I do, this is the code that I use - script: | dbx launch --job=standard_datatamers_lib-sample-integration-test-feature --trace displayName: 'Launch integration on test'

[renardeinside]

got it. seems like the integration test is not reporting a failed state after launching the tests. Let me make some fail triggering tests.

[renardeinside]

hi @Halpph !

It's more of an issue with unittest library - unfortunately, it will give 0 exit code even if tests failed.

Please use the following snippet to fix this behaviour an the end of your integration test:

if __name__ == "__main__":
    # please don't change the logic of test result checks here
    # it's intentionally done in this way to comply with jobs run result checks
    # for other tests, please simply replace the SampleJobIntegrationTest with your custom class name
    loader = unittest.TestLoader()
    tests = loader.loadTestsFromTestCase(SampleJobIntegrationTest)
    runner = unittest.TextTestRunner(verbosity=2)
    result = runner.run(tests)
    if not result.wasSuccessful():
        raise RuntimeError("One or multiple tests failed. Please check job logs for additional information.")

[renardeinside]

The snippet above is added to the cookie-cutter templated file. Please reopen the issue if the problem persists and thanks a lot for raising an issue!

[Halpph]

Thanks a lot! It definitely works now.