search

kubernetes-sigs / e2e-framework

A Go framework for end-to-end testing of components running in Kubernetes clusters.
Apache License 2.0
529 stars 103 forks source link

How to use BeforeEachTest #366

Closed shivaDS closed 4 months ago

shivaDS commented 10 months ago

`func TestMain(m *testing.M) { log.Println("Its main function") //testenv = env.New()

cfg, err := envconf.NewFromFlags()
if err != nil {
    log.Fatalf("failed to load env config: %s", err)
}

testenv = env.NewWithConfig(cfg)

testenv.Setup(
    envfuncs.InstallTanzuCli(),
)

testenv.Finish()

// testenv.BeforeEachTest(
//  envfuncs.InstallTanzuCliTestStep(),
// )

testenv.BeforeEachTest(func(ctx context.Context, cfg *envconf.Config, t *testing.T) (context.Context, error) {
    t.Log("Its before test function")
    return ctx, nil
})
// })
// testenv.BeforeEachTest(func(ctx context.Context, cfg *envconf.Config, t *testing.T) (context.Context, error) {
//  return createNSForTest(ctx, cfg, t, "5")
// })

os.Exit(testenv.Run(m))

}`

Here testenv.Setup() is getting executed but not BeforeEachTest. I am new to this framework. Could anyone please help?

vladimirvivien commented 10 months ago

Hi @shivaDS testenv.BeforeEachTest is executed right before defined Go test functions. So, for you to see BeforeEachTest executed, you need to define a Go test function, which I don't see in your example.

For instance, when you run the following example, the framework will execute your BeforeEachTest before it runs test function TestSomeCode:

var testenv env.Env

func TestMain(m *testing.M) {

    cfg, err := envconf.NewFromFlags()
    if err != nil {
    log.Fatalf("failed to load env config: %s", err)
    }

    testenv = env.NewWithConfig(cfg)

    testenv.Setup(
     envfuncs.InstallTanzuCli(),
    )

    testenv.Finish()

    testenv.BeforeEachTest(func(ctx context.Context, cfg *envconf.Config, t *testing.T) (context.Context, error) {
    t.Log("Its before test function")
    return ctx, nil
    })

    os.Exit(testenv.Run(m))
}

func TestSomeCode(t *testing.T){
    <insert-test-code>
}
shivaDS commented 10 months ago 
package log_test

import (
    "context"
    "envfuncs"

    //"log"
    "os"
    "testing"

    "sigs.k8s.io/e2e-framework/pkg/env"
    "sigs.k8s.io/e2e-framework/pkg/envconf"
)

var testenv env.Environment

func TestMain(m *testing.M) {

    testenv = env.New()

    testenv.Setup(
        envfuncs.InstallTanzuCli(),
    )

    testenv.Finish()

    testenv.BeforeEachTest(func(ctx context.Context, cfg *envconf.Config, t *testing.T) (context.Context, error) {
        t.Log("Its before test function")
        return ctx, nil
    })

    os.Exit(testenv.Run(m))
}

func TestSingleclusterInnerloopBasicSupplychainLocalSource(t *testing.T) {
    t.Log("its test case")
}
shivaDS commented 10 months ago 
module LogsPackage

go 1.19

require sigs.k8s.io/e2e-framework v0.3.0
shivaDS commented 10 months ago

hi @vladimirvivien I have added the complete code. Here the log in BeforeEachTest is not getting executed with both testing logger and "log" package logger

vladimirvivien commented 10 months ago

Hi @shivaDS Thanks for the your example. As I explained, the BeforeEachTest callback will get executed right before the TestSingleclusterInnerloopBasicSupplychainLocalSource test function is executed.

Specifically for your example, you should see "Its before test function" prior to "its test case". When you run the test, make sure to use the -v to see log output:

go test -v .

Otherwise, you will not see the t.Log output.

Hope that helps.

shivaDS commented 10 months ago

Hi @vladimirvivien , Thanks for your inputs. I am running using the command go test . -v Still I couldn't see that log.

package log_test

import (
    "context"
    "envfuncs"
    "os"
    "testing"

    "sigs.k8s.io/e2e-framework/pkg/env"
    "sigs.k8s.io/e2e-framework/pkg/envconf"
)

var testenv env.Environment

func TestMain(m *testing.M) {

    testenv = env.New()

    testenv.Setup(
        envfuncs.InstallTanzuCli(),
    )

    testenv.Finish()

    testenv.BeforeEachTest(func(ctx context.Context, cfg *envconf.Config, t *testing.T) (context.Context, error) {
        t.Log("Its before test function")
        return ctx, nil
    })

    os.Exit(testenv.Run(m))
}

func TestSingleclusterInnerloopBasicSupplychainLocalSource(t *testing.T) {
    t.Log("its test case")
}

// func setupTest(t *testing.T) func(t *testing.T) {
//  log.Println("setup test")

//  return func(t *testing.T) {
//      log.Println("teardown test")
//  }
// }

// func createNSForTest(ctx context.Context, cfg *envconf.Config, t *testing.T, runID string) (context.Context, error) {
//  ns := envconf.RandomName(runID, 10)
//  // ctx = context.WithValue(ctx, GetNamespaceKey(t), ns)

//  t.Logf("Creating NS %v for test %v", ns, t.Name())
//  // nsObj := v1.Namespace{}
//  // nsObj.Name = ns
//  return ctx, nil
// }

module LogsPackage

go 1.19

require sigs.k8s.io/e2e-framework v0.3.0

The output I get is,

dshiva@dshiva206HL log_test % go test -v . 2024/01/13 22:52:11 In env function === RUN TestSingleclusterInnerloopBasicSupplychainLocalSource main_test.go:36: its test case --- PASS: TestSingleclusterInnerloopBasicSupplychainLocalSource (0.00s) PASS ok LogsPackage/log_test (cached)

vladimirvivien commented 10 months ago

@shivaDS Did you figure this out? Do you think it's a bug ?

shivaDS commented 10 months ago

@vladimirvivien I am unable to make it work. Since I am new to golang and e2e-framework, I find it bit hard to understand why it is not working. So I cannot confirm whether it is really a bug

vladimirvivien commented 9 months ago

@shivaDS Apologies for the late reply as I was super busy. But, the following shows how the {Before,After}EachTestest works. For it to work you need the followings:

Hope this helps 

var testenv env.Environment

func TestMain(m *testing.M) {
    testenv = env.New()

    testenv.Setup(func(ctx context.Context, cfg *envconf.Config) (context.Context, error) {
        fmt.Println("** Setting up e2e test...")
        return ctx, nil
    })

    testenv.BeforeEachTest(func(ctx context.Context, cfg *envconf.Config, t *testing.T) (context.Context, error) {
        fmt.Printf(" --> Executing BeforeTest: test %s \n", t.Name())
        return ctx, nil
    })

    testenv.BeforeEachFeature(func(ctx context.Context, cfg *envconf.Config, t *testing.T, f features.Feature) (context.Context, error) {
        fmt.Printf("    > Executing BeforeFeature: %s \n", f.Name())
        return ctx, nil
    })

    testenv.AfterEachFeature(func(ctx context.Context, cfg *envconf.Config, t *testing.T, f features.Feature) (context.Context, error) {
        fmt.Printf("    > Executing AfterFeature: %s \n", f.Name())
        return ctx, nil
    })

    testenv.AfterEachTest(func(ctx context.Context, cfg *envconf.Config, t *testing.T) (context.Context, error) {
        fmt.Printf(" --> Executing AfterTest: %s \n", t.Name())
        return ctx, nil
    })

    testenv.Finish(func(ctx context.Context, _ *envconf.Config) (context.Context, error) {
        fmt.Println("** Finishing e2e test ")
        return ctx, nil
    })

    os.Exit(testenv.Run(m))
}

func TestSomething(t *testing.T) {

    // calls testenv.BeforeEachFeature here
    f1 := features.New("Feature 1").
        Assess("Assessment 1", func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
            return ctx
        })
    // calls testenv.AfterEachFeature here

    // calls testenv.BeforeEachFeature here
    f2 := features.New("Feature 2").
        Assess("Assessment 2", func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
            return ctx
        })
    // calls testenv.AfterEachFeature here

    // testevn.BeforeEachTest
    testenv.Test(t, f1.Feature(), f2.Feature())
    // testenv.AfterEachTest
}

When you run the previous code, you will get the following

go test -v .
** Setting up e2e test...
=== RUN   TestSomething
 --> Executing BeforeTest: test TestSomething 
    > Executing BeforeFeature: Feature 1 
=== RUN   TestSomething/Feature_1
=== RUN   TestSomething/Feature_1/Assessment_1
    > Executing AfterFeature: Feature 1 
    > Executing BeforeFeature: Feature 2 
=== RUN   TestSomething/Feature_2
=== RUN   TestSomething/Feature_2/Assessment_2
    > Executing AfterFeature: Feature 2 
 --> Executing AfterTest: TestSomething 
--- PASS: TestSomething (0.00s)
    --- PASS: TestSomething/Feature_1 (0.00s)
        --- PASS: TestSomething/Feature_1/Assessment_1 (0.00s)
    --- PASS: TestSomething/Feature_2 (0.00s)
        --- PASS: TestSomething/Feature_2/Assessment_2 (0.00s)
PASS
** Finishing e2e test 
ok      e2e-framework/workbench 0.679s
k8s-triage-robot commented 6 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 5 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 4 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 4 months ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-sigs/e2e-framework/issues/366#issuecomment-2216332139): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.