onsi / ginkgo

A Modern Testing Framework for Go
http://onsi.github.io/ginkgo/
MIT License
8.28k stars 654 forks source link

Run tests in individual process #1431

Open sagar-shah-uipath opened 3 months ago

sagar-shah-uipath commented 3 months ago

We have global variable defined as following

var (
    namespaceVal string
    once            sync.Once
    isSet           bool
)

func SetNamespace(namespace string) {
    if namespace == "" {
        panic("SetNamespace must be called with a non-empty string")
    }

    if namespace != "" && namespace != namespaceVal {
        panic("SetNamespace must be called with the same value")
    }

    once.Do(func() {
        namespaceVal = namespace
        isSet = true
    })
}

func IsNamespaceAlreadySet() bool {
    return isSet
}

func GetNamespace() string {
    if !IsNamepaceAlreadySet() {
        panic("SetNamespace must be called before calling GetNamespace")
    }

    return namespaceVal
}

We have various integration tests written for a different golang entrypoints command which calls SetNamespace internally to set value of namespace taken as input from integration test. When we run integration tests using ginkgo since everything is running parallel in single process trying to modify global variable defined above called namespaveVal, some of our tests running into error "SetNamespace must be called with the same value".

Is there anyway to ensure these integration tests don't interact with each other and ensure each run in separate process to avoid this issue.

Sample integration test code

    It("apply with namespace1", Serial, func() {
        ioStreams, _, _, _ := util.NewTestIOStreams()

        rootCmd := root.NewCommand(ioStreams)
        rootCmd.SetArgs([]string{"apply", inputPath, "--namespace", "namespace1"})

        err = rootCmd.Execute()
        Expect(err).To(BeNil())
    })

    It("apply with namespace2", Serial, func() {
        ioStreams, _, _, _ := util.NewTestIOStreams()

        rootCmd := root.NewCommand(ioStreams)
        rootCmd.SetArgs([]string{"apply", inputPath, "--namespace", "namespace2"})

        err = rootCmd.Execute()
        Expect(err).To(BeNil())
    })
onsi commented 3 months ago

hey @sagar-shah-uipath - Ginkgo is designed to handle your usecase - I’d suggest taking some time to read the documentation to familiarize yourself with how Ginkgo allows you to set up specs, how it expects your specs to be structured, how it runs them in parallel and to read more about some common Ginkgo patterns.

There are also several example Kubernetes test suites out there.

One thing to note: Ginkgo never runs your specs in parallel in a single process. It always runs specs in a given process serially and parallelization only occurs across multiple processes. The most common pattern I’ve seen is to set up a namespace for each test in a BeforeEach and tear it down in a DeferCleanup. The namespace can be a random guid or something that that uses GinkgoParallelProcess() to get a unique integer per process.

I’d like to encourage you to look through the docs and try organizing your tests around Ginkgo’s model for parallelization - I think you’ll find that the investment will pay off quite quickly :)