search

openshift / origin

Conformance test suite for OpenShift
http://www.openshift.org
Apache License 2.0
8.49k stars 4.7k forks source link

openshift-tests run throws fatal error for dry run when OPENSHIFT_SKIP_EXTERNAL_TESTS is not set #27975

Closed Rajalakshmi-Girish closed 1 year ago

Rajalakshmi-Girish commented 1 year ago 
+ openshift-tests run kubernetes/conformance --dry-run
  Jun  9 00:36:21.245: INFO: Enabling in-tree volume drivers
Attempting to pull tests from external binary...
  goroutine 1 [running]:
  runtime/debug.Stack()
    runtime/debug/stack.go:24 +0x6c
  github.com/openshift/origin/test/extended/util.FatalErr({0x17066aa0, 0xc00549d5f0})
    github.com/openshift/origin/test/extended/util/client.go:922 +0x3c
  github.com/openshift/origin/test/extended/util.(*CLI).AdminConfig(0x0?)
    github.com/openshift/origin/test/extended/util/client.go:750 +0x58
  github.com/openshift/origin/test/extended/util.(*CLI).AdminConfigClient(0xc005512f00)
    github.com/openshift/origin/test/extended/util/client.go:663 +0x30
  github.com/openshift/origin/pkg/test/ginkgo.extractBinaryFromReleaseImage({0x17a86a03, 0x9}, {0x17ab5b37, 0x12})
    github.com/openshift/origin/pkg/test/ginkgo/external.go:77 +0xa8
  github.com/openshift/origin/pkg/test/ginkgo.externalTestsForSuite({0x186bab70, 0x1bf4aa80})
    github.com/openshift/origin/pkg/test/ginkgo/external.go:33 +0x58
  github.com/openshift/origin/pkg/test/ginkgo.(*Options).Run(0xc000ce4fc0, 0x1b51f2c0, {0x17aa27ac, 0xf})
    github.com/openshift/origin/pkg/test/ginkgo/cmd_runsuite.go:168 +0x210
  main.newRunCommand.func1.1()
    github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:330 +0x298
  main.mirrorToFile(0x0?, 0x0?)
    github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:467 +0xf0
  main.newRunCommand.func1(0xc001275700?, {0xc00082eb60?, 0x4?, 0x17a77fdb?})
    github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:311 +0x88
  github.com/spf13/cobra.(*Command).execute(0xc000b83200, {0xc00082eb40, 0x2, 0x2})
    github.com/spf13/cobra@v1.6.0/command.go:916 +0x740
  github.com/spf13/cobra.(*Command).ExecuteC(0xc000b82f00)
    github.com/spf13/cobra@v1.6.0/command.go:1040 +0x370
  github.com/spf13/cobra.(*Command).Execute(...)
    github.com/spf13/cobra@v1.6.0/command.go:968
  main.main.func1(0xc000e86600?)
    github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:96 +0x8c
  main.main()
    github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:97 +0x498

panic: Your Test Panicked
github.com/openshift/origin/test/extended/util/client.go:923
  When you, or your assertion library, calls Ginkgo's Fail(),
  Ginkgo panics to prevent subsequent assertions from running.

  Normally Ginkgo rescues this panic so you shouldn't see it.

  However, if you make an assertion in a goroutine, Ginkgo can't capture the
  panic.
  To circumvent this, you should call

    defer GinkgoRecover()

  at the top of the goroutine that caused this panic.

  Alternatively, you may have made an assertion outside of a Ginkgo
  leaf node (e.g. in a container node or some out-of-band function) - please
  move your assertion to
  an appropriate Ginkgo node (e.g. a BeforeSuite, BeforeEach, It, etc...).

  Learn more at:
  http://onsi.github.io/ginkgo/#mental-model-how-ginkgo-handles-failure

goroutine 1 [running]:
github.com/onsi/ginkgo/v2.Fail({0xc00550e620, 0x20}, {0xc0063624d8?, 0x1?, 0x1?})
    github.com/onsi/ginkgo/v2@v2.9.1/core_dsl.go:374 +0x1e0
k8s.io/kubernetes/test/e2e/framework.Failf({0x17a76d8e?, 0xc000808480?}, {0xc006362538?, 0x1?, 0x1?})
    k8s.io/kubernetes@v1.27.1/test/e2e/framework/log.go:44 +0x6c
github.com/openshift/origin/test/extended/util.FatalErr({0x17066aa0, 0xc00549d5f0})
    github.com/openshift/origin/test/extended/util/client.go:923 +0xe0
github.com/openshift/origin/test/extended/util.(*CLI).AdminConfig(0x0?)
    github.com/openshift/origin/test/extended/util/client.go:750 +0x58
github.com/openshift/origin/test/extended/util.(*CLI).AdminConfigClient(0xc005512f00)
    github.com/openshift/origin/test/extended/util/client.go:663 +0x30
github.com/openshift/origin/pkg/test/ginkgo.extractBinaryFromReleaseImage({0x17a86a03, 0x9}, {0x17ab5b37, 0x12})
    github.com/openshift/origin/pkg/test/ginkgo/external.go:77 +0xa8
github.com/openshift/origin/pkg/test/ginkgo.externalTestsForSuite({0x186bab70, 0x1bf4aa80})
    github.com/openshift/origin/pkg/test/ginkgo/external.go:33 +0x58
github.com/openshift/origin/pkg/test/ginkgo.(*Options).Run(0xc000ce4fc0, 0x1b51f2c0, {0x17aa27ac, 0xf})
    github.com/openshift/origin/pkg/test/ginkgo/cmd_runsuite.go:168 +0x210
main.newRunCommand.func1.1()
    github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:330 +0x298
main.mirrorToFile(0x0?, 0x0?)
    github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:467 +0xf0
main.newRunCommand.func1(0xc001275700?, {0xc00082eb60?, 0x4?, 0x17a77fdb?})
    github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:311 +0x88
github.com/spf13/cobra.(*Command).execute(0xc000b83200, {0xc00082eb40, 0x2, 0x2})
    github.com/spf13/cobra@v1.6.0/command.go:916 +0x740
github.com/spf13/cobra.(*Command).ExecuteC(0xc000b82f00)
    github.com/spf13/cobra@v1.6.0/command.go:1040 +0x370
github.com/spf13/cobra.(*Command).Execute(...)
    github.com/spf13/cobra@v1.6.0/command.go:968
main.main.func1(0xc000e86600?)
    github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:96 +0x8c
main.main()
    github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:97 +0x498

This is after https://github.com/openshift/origin/pull/27570/commits/0060ccdd4d01b70980ee8b8108829e16b8714fa7

Looks like the line https://github.com/openshift/origin/blob/master/pkg/test/ginkgo/external.go#L77 panics as the config of cluster isnt' passed in above command. The dry run should ideally not need config of cluster to list tests.

Also the variable OPENSHIFT_SKIP_EXTERNAL_TESTS should be mentioned in the output of openshift-tests binary help output.

Rajalakshmi-Girish commented 1 year ago

@soltysh Can you please have a look.

openshift-bot commented 1 year ago

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

openshift-bot commented 1 year ago

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten /remove-lifecycle stale

openshift-bot commented 1 year ago

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen. Mark the issue as fresh by commenting /remove-lifecycle rotten. Exclude this issue from closing again by commenting /lifecycle frozen.

/close

openshift-ci[bot] commented 1 year ago

@openshift-bot: Closing this issue.

In response to [this](https://github.com/openshift/origin/issues/27975#issuecomment-1797991513): >Rotten issues close after 30d of inactivity. > >Reopen the issue by commenting `/reopen`. >Mark the issue as fresh by commenting `/remove-lifecycle rotten`. >Exclude this issue from closing again by commenting `/lifecycle frozen`. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.