search

onsi / ginkgo

A Modern Testing Framework for Go
http://onsi.github.io/ginkgo/
MIT License
8.38k stars 660 forks source link

Panicking in ReportAfterSuite #1467

Open dschveninger opened 1 month ago

dschveninger commented 1 month ago

In one of our Intergration test suite we are getting the following Panic in the ReportAfterSuite. Our code for executes but Ginkgo generated code panic. We have over 9 different test suite and this only happens in one of them.

Since it is integration test suite and requires many things, we can not run the code without the system so I cannot ship an example to you and our prototype do not fail like this. Can you see how it is failing and give us a area to look at for this type of failure?

 nc-api-testing-runner-268969422aae ] [ReportAfterSuite] SkipAnalysis Report
[ nc-api-testing-runner-268969422aae ] /app/tests/v20240701/clean-up-resources/clean_up_resources_suite_test.go:175
[ nc-api-testing-runner-268969422aae ]   > Enter [ReportAfterSuite] TOP-LEVEL - /app/tests/v20240701/clean-up-resources/clean_up_resources_suite_test.go:175 @ 10/17/24 01:50:31.427
[ nc-api-testing-runner-268969422aae ]   +--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[ nc-api-testing-runner-268969422aae ]   | Known Skips                                                                                                                                                        |
[ nc-api-testing-runner-268969422aae ]   +---+--------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
[ nc-api-testing-runner-268969422aae ]   | # | SKIP REASON                                                        | TEST CASE                                                                                 |
[ nc-api-testing-runner-268969422aae ]   +---+--------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
[ nc-api-testing-runner-268969422aae ]   | 1 | Skipping because virtual infrastructure resources were not created | Clean Up virtual infrastructure resources created by host Virtual Machine ARM Deployment  |
[ nc-api-testing-runner-268969422aae ]   +---+--------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
[ nc-api-testing-runner-268969422aae ]   | 2 | Spec skipped because Skip() was called in BeforeAll                | Clean Up virtual infrastructure ARM deployment                                            |
[ nc-api-testing-runner-268969422aae ]   +---+--------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
[ nc-api-testing-runner-268969422aae ]   | 3 | Spec skipped because Skip() was called in BeforeAll                | Clean Up virtual infrastructure delete the resource group used for virtual infrastructure |
[ nc-api-testing-runner-268969422aae ]   +---+--------------------------------------------------------------------+-------------------------------------------------------------------------------------------+
[ nc-api-testing-runner-268969422aae ]   < Exit [ReportAfterSuite] TOP-LEVEL - /app/tests/v20240701/clean-up-resources/clean_up_resources_suite_test.go:175 @ 10/17/24 01:50:31.427 (0s)
[ nc-api-testing-runner-268969422aae ] [ReportAfterSuite] PASSED [0.000 seconds]
[ nc-api-testing-runner-268969422aae ] ------------------------------
[ nc-api-testing-runner-268969422aae ] [ReportAfterSuite] Autogenerated ReportAfterSuite for --json-report --junit-report
[ nc-api-testing-runner-268969422aae ] autogenerated by Ginkgo
[ nc-api-testing-runner-268969422aae ]   > Enter [ReportAfterSuite] TOP-LEVEL - autogenerated by Ginkgo @ 10/17/24 01:50:31.427
[ nc-api-testing-runner-268969422aae ]   [PANICKED] Test Panicked
[ nc-api-testing-runner-268969422aae ]   In [ReportAfterSuite] at: /usr/local/go/src/runtime/panic.go:154 @ 10/17/24 01:50:31.462
[ nc-api-testing-runner-268969422aae ] 
[ nc-api-testing-runner-268969422aae ]   runtime error: slice bounds out of range [834:0]
[ nc-api-testing-runner-268969422aae ] 
[ nc-api-testing-runner-268969422aae ]   Full Stack Trace
[ nc-api-testing-runner-268969422aae ]   < Exit [ReportAfterSuite] TOP-LEVEL - autogenerated by Ginkgo @ 10/17/24 01:50:31.462 (35ms)
[ nc-api-testing-runner-268969422aae ] [ReportAfterSuite] [PANICKED] [0.035 seconds]
[ nc-api-testing-runner-268969422aae ] [ReportAfterSuite] Autogenerated ReportAfterSuite for --json-report --junit-report
[ nc-api-testing-runner-268969422aae ] autogenerated by Ginkgo
[ nc-api-testing-runner-268969422aae ] ------------------------------
[ nc-api-testing-runner-268969422aae ] 
[ nc-api-testing-runner-268969422aae ] Summarizing 1 Failure:
[ nc-api-testing-runner-268969422aae ]   [PANICKED!] [ReportAfterSuite] Autogenerated ReportAfterSuite for --json-report --junit-report
[ nc-api-testing-runner-268969422aae ]   /usr/local/go/src/runtime/panic.go:154
[ nc-api-testing-runner-268969422aae ] 
[ nc-api-testing-runner-268969422aae ] Ran 2 of 5 Specs in 79.082 seconds
[ nc-api-testing-runner-268969422aae ] FAIL! -- 2 Passed | 0 Failed | 0 Pending | 3 Skipped
dschveninger commented 1 month ago

ginkgopanicspec.json Here is the json output that is causing the panic Hope this helps.

dschveninger commented 3 weeks ago

WE were able to fix it by stop using the --junit-report switch. We are still trying to create a sample test suite to show it.

onsi commented 3 weeks ago

hey sorry for the radio silence i've been swamped these days. if it's a panic in ginkgo then it's probably being swallowed by ginkgo. try running with the environment ment variable "GINKGO_PRUNE_STACK=FALSE" and see if you get more in the stack trace

dschveninger commented 3 weeks ago

No problem thank you for that we will do that and report back.