Open aws-sde opened 3 weeks ago
@aws-sde Since you seem to have spent a good amount of time trying to debug this issue, would you like to raise a pull request to get this fixed? We can help you with the review and merge
Yes, I'll try to find time to submit a fix hopefully next weekend.
TestNG Version
7.10.2
Expected behavior
All failed or skipped configuration methods should have a throwable set on their test result.
Actual behavior
Only the first failed or skipped configuration method has a throwable set in its test result. Subsequent configuration methods do get skipped (due to the initial failure/skip), but they do not set the throwable.
This has the consequence of intermittently leaving the throwable
null
on test results for skipped test methods. WhenTestInvoker
detects a configuration failure, it marks the test method as skipped, and sets its throwable to be the configuration method's throwable. There are two problems with this:ConcurrentHashMap
, which does not guarantee ordering.Exception.getExceptionDetails
naïvely uses the throwable of the first match for a configuration method's test result.Is the issue reproducible on runner?
Test case sample
I'm not able to reproduce
onTestSkipped
showing anull
throwable with this, but it absolutely happens in my actual code base. In any case, I think this sufficiently demonstrates that the throwable isnull
for the other configuration.