An option added to keep only the latest test result for each test cas…

MobileNativeFoundation / bluepill

Bluepill is a reliable iOS testing tool that runs UI tests using multiple simulators on a single machine

BSD 2-Clause "Simplified" License

3.18k stars 231 forks source link

An option added to keep only the latest test result for each test cas… #467

Open saim80 opened 3 years ago

saim80 commented 3 years ago

…e are kept. So, retry attempts will end up with one failure or pass result.

In our organization, we do not want to have repeated test results which made us confused via Jenkins JUnit Test Result page.

I added an option to keep only the latest result for each test case. So, this will keep us informed of either one pass or failure result only without duplicate test results when retry is attempted.

ravimandala commented 3 years ago

Good to have, but my hesitation is we are adding one more flag. @saim80 Instead can you do some post-processing over the Bluepill report to trim what you don't need?

chenxiao0228 commented 3 years ago

@saim80 thanks for your contribution!

Come to think of it, we do not really need coverage data for the failed attempts, like you said it does not provide extra value but confusion. I think we are good to make it the default behavior and do not add the new flag.

A couple of things I'd like to ask though

can you figure out why one of the checks failed?
can you add a unit test case for the de-dup behavior?

Many thanks!

cccCody commented 2 years ago

@chenxiao0228

"we are good to make it the default behavior"

Did this happen? Is this actually the default behavior now?