onsi / ginkgo

A Modern Testing Framework for Go
http://onsi.github.io/ginkgo/
MIT License
8.4k stars 660 forks source link

Doesn't show failed test case when -p option is set. #1303

Open bitoku opened 1 year ago

bitoku commented 1 year ago
❯ ginkgo version                                             
Ginkgo Version 2.13.1

When these conditions are met, failed test case are not shown.

You will be able to reproduce this issue in my test repository.

https://github.com/bitoku/ginkgo-test

❯ ginkgo        
--- FAIL: TestAdd (0.00s)
    main_test.go:7: Expected 1 + 1 to equal 2
Running Suite: Ginkgo Suite - /Users/atokubi/workspace/ginkgo
=============================================================
Random Seed: 1700457476

Will run 1 of 1 specs
•

Ran 1 of 1 Specs in 0.000 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
FAIL

Ginkgo ran 1 suite in 1.540507723s

Test Suite Failed
❯ ginkgo -p
Running Suite: Ginkgo Suite - /Users/atokubi/workspace/ginkgo
=============================================================
Random Seed: 1700457506

Will run 1 of 1 specs
Running in parallel across 11 processes
•

Ran 1 of 1 Specs in 0.066 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped

Ginkgo ran 1 suite in 1.218479444s

Test Suite Failed

I know it is not supposed to have both ginkgo spec files and builtin unittest files, but it is very confusing when people want to migrate builtin unittest to ginkgo.

blgm commented 1 year ago

It looks like Ginkgo is running the test and catching that it failed, but it's just swallowed all information about the test. Even the verbose options don't help:

$ ginkgo -p -vv --show-node-events

Running Suite: Ginkgo Suite - /tmp/tt/ginkgo-test
=================================================
Random Seed: 1700523524

Will run 1 of 1 specs
Running in parallel across 7 processes
------------------------------
• [0.000 seconds]
Add
/tmp/tt/ginkgo-test/ginkgo_test.go:8
  adds two numbers
  /tmp/tt/ginkgo-test/ginkgo_test.go:9

  Timeline >>
  > Enter [It] adds two numbers - /tmp/tt/ginkgo-test/ginkgo_test.go:9 @ 11/20/23 23:38:45.485
  < Exit [It] adds two numbers - /tmp/tt/ginkgo-test/ginkgo_test.go:9 @ 11/20/23 23:38:45.485 (0s)
  << Timeline
------------------------------

Ran 1 of 1 Specs in 0.009 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped

Ginkgo ran 1 suite in 1.177801911s

Test Suite Failed

For what it's worth, ideally this should work better. It's a reasonably common requirement to mix TestXYZ() style tests with Ginkgo tests, especially for ExampleABC() documentation tests.

XuQinWL commented 3 months ago

@blgm actually,when i run my bash script:

#!/usr/bin/env bash
set -ex

cd "$(dirname "$0")"

WORKDIR="$(pwd)"

while getopts ":c:l:" opt; do
  case $opt in
    c) CONFIG_FILE="$OPTARG"
    ;;
    l) CUSTOM_LABEL="$OPTARG"
    ;;
    \?) echo "Invalid option -$OPTARG" >&2
    exit 1
    ;;
  esac
done

if [ -z "${CONFIG_FILE}" ]; then
  echo "config file empty"
  exit 1
fi

ginkgo -v --show-node-events --trace --label-filter="${CUSTOM_LABEL}" init/... -- -c ${WORKDIR}/configs/${CONFIG_FILE}

ginkgo told me :--show-node-events provide but not defined Could you please tell me how to solve this problem