search

onsi / ginkgo

A Modern Testing Framework for Go
http://onsi.github.io/ginkgo/
MIT License
8.23k stars 650 forks source link

Documentation for running Ginkgo outside of `go test` #1451

Open dagan opened 3 weeks ago

dagan commented 3 weeks ago

I have a use case for running Ginkgo tests without go test, and I am curious if any documentation exists for doing that.

I've perused the code base and see that the bulk of test management falls under the internal package, which I can certainly understand. Is there something I've missed that exposes the ability to programmatically launch Ginkgo outside of a go test context?

For my specific use case, I'm hoping to utilize Ginkgo's DSL for defining and organizing tests, the ability to filter tests by labels, and of course the reporters, but all integrated within a larger application with use cases separate from Ginkgo's. At this point, I suspect that compiling the tests and launching them independently is the move viable path forward, but I figured I'd at least ask. :-)

onsi commented 3 weeks ago

hey hey @dagan - this has come up a couple of times in the past. long story short: your best bet is to compile the tests (or ship with a precompiled test binary) an orchestrate running them by shelling out and parsing the JSON reporter format.

Longer answer: given you’ve looked at some of the code you can see that there are ways to get the DSL to run in-process (the internal_integration tests, in particular, provide a pattern for doin this). And I can imagine exposing a mechanism for letting users leverage this themselves. But there will be caveats - chief among them: because of its magic use of global Ginkgo is not thread-safe. Moreover, its (re)use of closures to define the test hierarchy means that test pollution will occur if you try to run the tests in parallel in the same process. This is why Ginkgo orchestrates multiple processes to run in parallel. If you want to be able to run in parallel you’ll want to use the Ginkgo CLI and offload all that complexity to Ginkgo. If you’re ok with just running in series, though then, yeah, there could be options.

But I think you’ll find that a shelling out to a (potentially precompiled) test suite will largely Just Work™ and be good enough for your usecase.

dagan commented 3 weeks ago

Thank you for the response, @onsi. I appreciate the time (and very sincerely appreciate the effort that has gone into Ginkgo and Gomega). ✌️