cucumber / godog

Cucumber for golang
MIT License
2.32k stars 254 forks source link

Running with concurrency shouldn't intermingle the test outputs #643

Closed thomshutt closed 2 weeks ago

thomshutt commented 1 month ago

👓 What did you see?

Example output showing three different test's steps being mixed together image

✅ What did you expect to see?

It would be more readable and easier to debug to hold onto a test's output until it's complete and then write it all at once.

📦 Which tool/library version are you using?

No response

🔬 How could we reproduce it?

Set Concurrency: 2 (or more) in the config

📚 Any additional context?

No response

Johnlon commented 1 month ago

The suggestion is one possibility, but would result in silence for protracted periods.

The reality of the situation is that folk will use things like fmt.* and even the root println fn, and unlike the JVM sadly golang doesn't provide a reliable way to intercept stdout/stderr to capture that output to defer logging. I tried it ages ago and it doesn't work and there is nothing the framework can really do to fix that.

What does work (for godog and other MT code) is to create a logger for each goroutine (eg in the before scenario), and to pass that logger to each step via the context object and then that logger can aggregate output until the after scenario where it could be logged en-masse. This is something you can do yourself.

This approach also has advantages as you can attache the captured console to the steps as you go along so that it's available in HTML report and so on.

tigh-latte commented 1 month ago

@Johnlon I've Implemented a possible approach to fix this without interrupting test execution in #645, if you have time to take a look/give feedback

Johnlon commented 1 month ago

does it only deal with the formatters or also ad-hoc console output from println, fmt* and loggers, which for me has been the main interest.

tigh-latte commented 1 month ago

Just formatters. Should godog deal with adhoc printing to stdout outside of its domain? I don't think so myself. If I want ordered logging in a concurrent library, then I ought to learn how to use the logging of that library.