onsi / ginkgo

A Modern Testing Framework for Go
http://onsi.github.io/ginkgo/
MIT License
8.23k stars 650 forks source link

reporting flags do not support absolute paths, unclear from flag description/docs. #1001

Open kurt-google opened 2 years ago

kurt-google commented 2 years ago

It seems that none of the reports flags (e.g. junit-report) support absolute paths and ginko generally just appends whatever you input to output-dir but this isn't very clear from the flags or documentation that I found. While I would appreciate absolute paths being supported if thats not possible can the flags/documentation reflect this or perhaps log a warning if an absolute path is used for these output file flags?

onsi commented 2 years ago

hey @kurt-google - I should definitely update the docs. Since Ginkgo can run multiple suites in one go the intent is for you to let Ginkgo manage where the files end up with the idea being that --junit-report is the name of the file and --output-dir is how you control where the location of the file ends up. when there are multiple suites a single consolidated report is written to that output-dir with that junit-report filename but you can also specify --keep-separate-reports to have files named ${SUITENAME}-${junit-report} stored in output-dir. The docs here are trying to convert this but they're clearly failing!!

kurt-google commented 2 years ago

when there are multiple suites a single consolidated report is written to that output-dir with that junit-report filename

Thanks this is what I have been using (for outputing ginkgo results to bazel's XML_OUTPUT_FILE). But having to redirect all output via --output-dir means ginkgo might output something that overwrites what bazel was going to put there or otherwise conflict, and that I need to split this expected output into a dir/file parts instead of just passing it along.

For example my preference would be to point --output-dir at $TEST_UNDECLARED_OUTPUTS_DIR so i can capture all of the ginkgo output (results in a zip file for analysis later), but direct --junit-report to $XML_OUTPUT_FILE so that ginkgo junit output integrates seamlessly with the junit infrastructure around bazel.