onsi / ginkgo

A Modern Testing Framework for Go
http://onsi.github.io/ginkgo/
MIT License
8.22k stars 650 forks source link

Can only see 1 It() container logs when running in an Ordered Describe() container #1269

Closed Mo0rBy closed 1 year ago

Mo0rBy commented 1 year ago

First of all, here is my Ginkgo testing code:

var _ = Describe("APP services:", Label("template-test", "service"), func() {
    for _, microService := range microServices {
        generateServiceTests(microService)
    }
})

func generateServiceTests(microService MicroService) {
    var service corev1.Service

    Describe(fmt.Sprintf("The '%s' microservice service", microService.Name), Ordered, ContinueOnFailure, func() {
        BeforeAll(func() {
            helmChartPath := fmt.Sprintf("../charts/%s", microService.Name)
            renderedOutput := helm.RenderTemplate(GinkgoT(), &helm.Options{}, helmChartPath, microService.ReleaseName, []string{"templates/service.yaml"})
            helm.UnmarshalK8SYaml(GinkgoT(), renderedOutput, &service)
        })

        It("should have the correct name", func() {
            Expect(service.Name).To(Equal(microService.ReleaseName))
        })

        It("should have the correct selector", func() { // This is what I cannot see in the logs
            expectedSelector := map[string]string{ "app": microService.ReleaseName }
            Expect(service.Spec.Selector).To(Equal(expectedSelector))
        })
    })
}

This code is simply using the TerraTest library to render Helm chart templates, map them to objects in the kubernetes/client-go library, and then make assertions on the rendered helm templates to ensure they work correctly. I have several microservices, so I have an array of a struct that is passed into this code which loops over that array and executes the tests for each object in that array. I prefer this way instead of using DescribeTable() as I can use the same list of objects in multiple locations.

Anyway, I am using a BeforeAll() to create my rendered Helm template object and then I want to perform multiple assertions in individual It() containers. My issue is that it seems as if the 2nd It() is executing, but I cannot see it logging anything to my terminal. I believe it is executing as I can see 2 green dots, not just 1, and I have Ran 12 specs which makes sense as I have 2 It() containers for 6 different microservice objects, but again, I can only see the logs of the 1st It() container.

Here is my terminal output. Sorry about having to hide a large amount of the text, but you can still see what I am describing very easily: Screenshot 2023-08-30 at 12 22 36 copy

onsi commented 1 year ago

hey @Mo0rBy - what is actually generating this logging? I don't see any print/etc. statements in your second It and it doesn't look like you are running with ginkgo -v. My best guess is that it's something in the BeforeAll() - most likely helm.UnmarshalK8SYaml which is getting the name of the test from the GinkgoT() and logging it. Since you are using a BeforeAll this will only run once - and it will happen to be associated with the first test in the Ordered container, namely "...it should have the correct name...".

If you run with ginkgo -v I suspect you'll see the "...correct selector..." tests too. Ditto if you simply add a fmt.Println("foo") in the second It.

Mo0rBy commented 1 year ago

Ah yes, you are correct @onsi , thank you. The helm.UnmarshalK8SYaml is producing the output. Using the -v flag shows the spec I am looking for. Thanks again! Closing this issue.

Mo0rBy commented 1 year ago

Is there anyway to make sure any fmt.Println()'s or any other outputs are produced before the name of the test is in the logs? Maybe a nice improvement? But probably would not be useful for many people.

onsi commented 1 year ago

no there is no way to do that - can you share more about what problem you are trying to solve and i can try to help?

Mo0rBy commented 1 year ago

I just want the output of the helm.UnmarshalK8SYaml in the BeforeAll to not have the first It() test name by it, I would rather it print the text for the Describe block it is within. But using the -v is fine, it prints the output once (with the first It() test name) and then it prints the 2nd It() more verbosely, which is just fine for me.

It's a Helm template test, so it is very useful to see the actual printed yaml that the template produces, so if the test fails, you can see exactly what the output of the Helm template is and see WHY it failed. Having the test name of the first It() next to the Helm template output is a very minor thing. I was more concerned with only being able to see the 1st It() in the log, and thinking that I should be able to see the 2nd It(), althought it doesn't produce any output, I thought I should be able to see the test name > pass/fail, and with -v I can see just that.