search

gotestyourself / gotestsum

'go test' runner with output optimized for humans, JUnit XML for CI integration, and a summary of the test results.
Apache License 2.0
1.99k stars 118 forks source link

Need Help with rerun-fails #385

Closed lovefromindia closed 6 months ago

lovefromindia commented 6 months ago

thanks for gotestsum package

We use below command on our Build but rerun didn't worked as expected.

gotestsum --rerun-fails --packages="./..." --format testname --junitfile ../../jsonreport.xml -- -coverprofile=../../cover.out ./... -p 7 -timeout 0

Is it because of using -p flag for parallel testcases? Any test case which failed only ran once and not twice as expected!

dnephin commented 6 months ago

Hello! Thank you for the bug report.

The only thing I notice from the command you shared is that ./... appears twice. Once in as the value for --packages , and again after cover.out. It is correct to pass that value to --packages, but I think you need to remove the second one after cover.out. I'm not sure if that is causing the problem or if it's something else.

The -p flag should be ok, I haven't seen problems with that before.

Can you share some of the test output? How many tests are failing, and are any failing with a panic?

lovefromindia commented 6 months ago

Thanks for the response. We already removed the twice occuring packages and then also the rerun didn't worked For panic part, I ensure no panic occured and assertions failed for the test case but I expected rerun for the test cases that failed. That didn't occured Also, for argument -packages do we need to list all packages instead of recursive package path "./..." ? With -count=2 all test cases ran twice as expected but not with only rerun-fails

I just wanted to know if I am doing anything wrong with command that my script generates to run at build environment?

gotestsum --rerun-fails --packages="./..." --format testname --junitfile ../../jsonreport.xml -- -coverprofile=../../cover.out -p 7 -timeout 0

dnephin commented 6 months ago

Can you try with adding the --debug flag to gotestsum ? That will print out the go test commands that are run each time. That should help us understand why there is no rerun.

If you can share some output of the test run that may help us debug, but from what you've shared I don't see anything that would prevent the re-runs.

lovefromindia commented 6 months ago

Thanks for the quick response, dnephin I thinks issue was with the argument of packages="./...". Instead we list out all packages in double quotes e.g. "a b c" and now reruns works as expected. Now the coverage report generated is of last run. Is there any argument that gotestum give so that i can get combined coverage report of all runs?

dnephin commented 6 months ago

Ah, yes there is a way to make that work. See https://github.com/gotestyourself/gotestsum/issues/274#issuecomment-1321222920, and this demo repo: https://github.com/neiser/gotestsum-rerun-dependent-subtests

lovefromindia commented 6 months ago

Thanks for sharing. I am closing this issue and also apologise that i bothered you with too much questions and so kind of you to answer them.