Test cases that run as part of our regression build need to have a defined method for supplying error information in the case of a failed test. A human might be able to get this error detail by looking at stdout, stderr, or digging through logs, but the test case should be able to provide a "best guess" error that indicates the underlying cause of the failure. This is used by cognitive systems to match other failure to existing issues.
For example, a given test may fail due to a NullPointerException with a specific stack trace. We need a system so that the test case can place that stack trace in a specific location that is available to the results aggregator and which can be fed to the cognitive analysis system. The cognitive analysis can then compare that stack trace against error information in existing issues to be able to identify if this particular failure is a known issue.
marikaj123
commented
4 years ago
Test cases that run as part of our regression build need to have a defined method for supplying error information in the case of a failed test. A human might be able to get this error detail by looking at stdout, stderr, or digging through logs, but the test case should be able to provide a "best guess" error that indicates the underlying cause of the failure. This is used by cognitive systems to match other failure to existing issues.
For example, a given test may fail due to a NullPointerException with a specific stack trace. We need a system so that the test case can place that stack trace in a specific location that is available to the results aggregator and which can be fed to the cognitive analysis system. The cognitive analysis can then compare that stack trace against error information in existing issues to be able to identify if this particular failure is a known issue.