hrcorval / behavex

BDD testing solution designed to enhance your Behave-based testing workflows
https://github.com/hrcorval/behavex
MIT License
89 stars 20 forks source link

how this behavex being invoked in jenkins job ? #52

Closed chunji08 closed 1 year ago

chunji08 commented 1 year ago

I was able to run bahavex in my jenkins job, but the job results are always of "blue" icon, this is quite different from my previous running, as this "blue" icon means the test cases are 100% succeeded, but this time, I guess it is for the execution is 100% finished, but however the result looks, it would be within the report summary.

Anyway, my extra impression on this tool is:

  1. It is much faster to have the test cases executed with behavex.
  2. I have seen extra test cases failure during the run-time, which I don't understand, because the test cases I have selected does not have depends on each other. it seems more frequent intermit failures to me.

Thanks,

Chun

hrcorval commented 1 year ago

Hi @chunji08, thanks for posting this so I have the possibility of providing an answer. Basically, when executing BehaveX from a build server like Jenkins, you need to configure the plan to read the provided JUnit reports that are being generated in the output folder, into the following path: \behave The framework generates one XML file (with JUnit format) per scenario or feature that Jenkins will be reading at runtime and will provide partial feedback while the execution is being performed.

Regarding the extra test cases failure, we have seen both cases: 1 The system under test gets stressed with parallel test executions and everything starts getting slower or timed out, causing scenarios to fail 2 The system where the tests are being executed is getting out of memory/CPU (any of the above usually cause you need to reduce the number of parallel instances)

I hope this helps :)