Discovery - Test statistics and history

[JunTaoLuo]

User Story or Problem Statement

As a Developer or DevOps engineer who was notified of a test failure, I would like to know the history of the failing test and statistics about the types of test failures to better diagnose and fix the underlying issues.

Description or Additional Context

See Slack discussion. Though in this case, the concern was with flaky tests, there are a variety of concerns such as test suite health and reliability.

Steps for Implementation

This issue involves looking at our current test reporting mechanisms and identifying areas for improvements (e.g. what kind of statistics and artifacts can be reported/preserved for future analysis) and mechanisms for aggregation and reporting. This may involve DataDog or other platforms and tools.

Acceptance Criteria

• [ ] TBD

Team

Please check the team(s) that will do this work.

• [x] CMS Team
• [ ] Public Websites
• [ ] Facilities
• [ ] Accelerated Publishing

[JunTaoLuo]

cc @maortiz-27-80 @BerniXiongA6 still very bare bones but here's an issue to track our test statistics/reporting improvements.