Summary not showing what failed

[lucacome]

I have random failures like this

There were failures detected in the following suites:
  static ./internal/mode/static

Test Suite Failed

And this it's all it says, I don't see any failures in the output or a summary of what failed or any errors. Should there be something in the summary? Is there any way to also force the output to show the errors?

I also tried to use the JSON reporting, but I can't find a tool that supports it to analyze it. Any suggestions? The docs say that that's format with all the info but maybe I should just switch to JUnit?

[onsi]

hey @lucacome - there should be plenty more output than that. Is there any more that you can share? Is the code open sourced so I can take a look? How are you running the suite? What versions of ginkgo and go are you using, and on which platform?

The JSON report should have everything - but there should also be much more command line output - so there's something off either with the setup or usage that i'd be happy to help figure out.

[lucacome]

Hey @onsi yes the repo is open source https://github.com/nginxinc/nginx-gateway-fabric and this specific problem is in our unit tests that we run with this command https://github.com/nginxinc/nginx-gateway-fabric/blob/79636f4f4b8b89b48c167f1245b85768df967c5b/Makefile#L196

go run github.com/onsi/ginkgo/v2/ginkgo --randomize-all --randomize-suites --race --keep-going --fail-on-pending --fail-fast --trace --covermode=atomic --coverprofile=coverage.out --force-newlines $(GITHUB_OUTPUT) -p -v -r internal

You can see one of the failures that doesn't have any other output here https://github.com/nginxinc/nginx-gateway-fabric/actions/runs/11019924001/job/30603633017?pr=2522

I'm working on uploading the reports to the PRs (but still haven't figured out a way to parse the JSON one).

Thanks for your help!

[onsi]

hey - so the unit tests under internal/mode/static are a mix of standard go tests and ginkgo tests and you are running in parallel. This isn't a very well supported test configuration - particularly given Ginkgo's approach to parallelization is quite different to Go's testing. Best I can recommend is to either switch to all-Ginkgo for this suite or switch to all-Go.

To say a bit more: Ginkgo's spinning up a bunch of processes to run in parallel. And each process will invoke the go tests as well as its subset of Ginkgo tests. The output of the go tests isn't going to be correctly managed and collated from these multiple processes.

[lucacome]

Thanks! We had a feeling that could cause some problems 🙁 Is there a way to tell ginkgo to only select gingko tests? I couldn't find anything in the docs, but hopefully I missed something.

[onsi]

Sadly no - Go test is gonna run those tests and Ginkgo doesn't have a way to tell it not to. I'd recommend converting to one or the other within a given suite.

[lucacome]

Thanks, I'll close this for now and report back if this happens again after we convert the tests.