Open bjchambers opened 1 year ago
I think we should make a small CLI tool for running the most of the commands currently in the Makefile.
Options on commands would be much more straightforward than enumerating every potential command in the Makefile.
I'm 50/50 -- I think it may be nice to just document the commands and people can create aliases if they want? Specifically, it seems easier to "know" how things are being run if we just document "use the ginkgo runner". But :shrug:
Summary
When running locally, it would be useful if we could rely on the labels to filter integration tests. For instance
--query-filter='!pulsar && !redis'
to run all of the tests other than Pulsar and Redis tests.We already have many of the tests properly labeled. For example: https://github.com/kaskada-ai/kaskada/blob/main/tests/integration/api/mat_pulsar_to_obj_store_test.go#L28
However, some of the tests rely on Pulsar without being labeled. For example: https://github.com/kaskada-ai/kaskada/blob/main/tests/integration/api/queries_v1_test.go#L36
In general, we should prefer to label entire functions as we currently do. But based on the Gingkgo docs, there are other options if we really need these tests that use Pulsar internally:
(from https://onsi.github.io/ginkgo/#spec-labels)
The other part of this is that the current structure of the makefile makes it difficult to pass arguments -- we need to create additional targets for every change. We should look at passing other arguments through the makefile, or using a shell script or something. Then we could more easily run something like
ENV=local-local go run github.com/onsi/ginkgo/v2/ginkgo ./... --label-filter='!pulsar && !redis'
(note that this also omits the-vv
since the extra verbosity is just spammy in many cases).