Open amazingankur opened 5 years ago
A build step fails if any of your commands return a non-zero exit code.
I assume cucumber is returning non-zero if there are failed tests? Are you by any chance wrapping your call to cucumber in some bash script that doesn't have set -e
set and are running commands after the call to cucumber?
In general, if you are invoking bash scripts from a pipeline command step, you ought to set -e
to make it bail out if any command fails. You'll see this at the top of most bash scripts within build pipelines:
#!/bin/bash
set -ueo pipefail
<stuff>
This means:
I am running simple sh commands like git clone/pull followed by mvn test In code i am running below: KarateStats stats = CucumberRunner.parallel(getClass(), 10, karateOutputPath); Above line provides the number of failures in cucumber. Not sure how this input can be read by build I am failrly new to build pipelines so excuse me for obvious/silly questions
I just wanted to highlight with my response that there isn't too much special about how Buildkite handles things. If you have a step like this for example:
steps:
- label: run test
command:
- git clone some stuff
- mvn test
If the mvn test
command returns a non-zero exit code, then this step will fail and hence mark your build as a failure.
The important thing is that if you want a step to fail, you need to make sure a command in your step returns a non-zero exit code. So I'd start debugging this locally and try to get it to a point where your script that invokes the tests does indeed return a non-zero exit code when tests fail.
Thanks much! Above answer gives me a start. I'll just post once I am finished.
Working fine once i put Assert on test results for failed test cases. Thanks and appreciate the help.
Buildkite passes the build even though cucumber tests get failed. Any help is appreciated.