github-actions[bot]
commented
3 months ago Conformance comparison report-Cross Engine
| Base (eval) | legacy | +/- | |
|---|---|---|---|
| % Passing | 91.22% | 92.59% | 1.37% |
| :white_check_mark: Passing | 5341 | 5420 | 79 |
| :x: Failing | 514 | 434 | -80 |
| :large_orange_diamond: Ignored | 0 | 0 | 0 |
| Total Tests | 5855 | 5854 | -1 |
Number passing in both: 5102
Number failing in both: 195
Number passing in eval engine but fail in legacy engine: 239
Number failing in eval engine but pass in legacy engine: 319 :interrobang: CONFORMANCE REPORT REGRESSION DETECTED :interrobang: The complete list can be found in GitHub CI summary, either from Step Summary or in the Artifact. 319 test(s) were failing in eval but now pass in legacy. Before merging, confirm they are intended to pass. The complete list can be found in GitHub CI summary, either from Step Summary or in the Artifact.
Relevant Issues
Description
Other Information
Updated Unreleased Section in CHANGELOG: [NO]
Any backward-incompatible changes? [NO]
Any new external dependencies? [NO]
Do your changes comply with the Contributing Guidelines and Code Style Guidelines? [YES]
License Information
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.