partiql / partiql-lang-kotlin

PartiQL libraries and tools in Kotlin.
https://partiql.org/
Apache License 2.0
536 stars 60 forks source link

Add binary API compatibility checks to v1 branch #1427

Closed alancai98 closed 2 months ago

alancai98 commented 2 months ago

Relevant Issues

Description

PR shows the current set of public APIs in all of our published libraries. I used https://github.com/Kotlin/binary-compatibility-validator gradle plugin which maintains a list of public binary APIs (as a .api file committed to our project). By default, the gradle task will run the apiCheck task. If there's a change to the public API (adding, modifying, deleting anything public), the apiCheck task will fail. To fix the failure, we can run apiDump to update the .api file(s) and commit the changes.

Other Information

License Information

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

github-actions[bot] commented 2 months ago

Conformance comparison report-Cross Engine

Base (legacy) eval +/-
% Passing 92.52% 82.12% -10.40%
:white_check_mark: Passing 5383 4778 -605
:x: Failing 435 1040 605
:large_orange_diamond: Ignored 0 0 0
Total Tests 5818 5818 0

Number passing in both: 4606

Number failing in both: 263

Number passing in legacy engine but fail in eval engine: 777

Number failing in legacy engine but pass in eval engine: 172 :interrobang: CONFORMANCE REPORT REGRESSION DETECTED :interrobang: The complete list can be found in GitHub CI summary, either from Step Summary or in the Artifact. 172 test(s) were failing in legacy but now pass in eval. Before merging, confirm they are intended to pass. The complete list can be found in GitHub CI summary, either from Step Summary or in the Artifact.

Conformance comparison report-Cross Commit-LEGACY

Base (04e33a8c716eac240a33278bf87b0ba254411c67) 38487f6f5eeb3e9aa7a6e13b57e6e3bcbe24a6d1 +/-
% Passing 92.51% 92.52% 0.02%
:white_check_mark: Passing 5382 5383 1
:x: Failing 436 435 -1
:large_orange_diamond: Ignored 0 0 0
Total Tests 5818 5818 0

Number passing in both: 5382

Number failing in both: 435

Number passing in Base (04e33a8c716eac240a33278bf87b0ba254411c67) but now fail: 0

Number failing in Base (04e33a8c716eac240a33278bf87b0ba254411c67) but now pass: 1 The following test(s) were previously failing but now pass. Before merging, confirm they are intended to pass:

Click here to see - MYSQL_SELECT_29, compileOption: LEGACY

Conformance comparison report-Cross Commit-EVAL

Base (04e33a8c716eac240a33278bf87b0ba254411c67) 38487f6f5eeb3e9aa7a6e13b57e6e3bcbe24a6d1 +/-
% Passing 82.12% 82.12% 0.00%
:white_check_mark: Passing 4778 4778 0
:x: Failing 1040 1040 0
:large_orange_diamond: Ignored 0 0 0
Total Tests 5818 5818 0

Number passing in both: 4778

Number failing in both: 1040

Number passing in Base (04e33a8c716eac240a33278bf87b0ba254411c67) but now fail: 0

Number failing in Base (04e33a8c716eac240a33278bf87b0ba254411c67) but now pass: 0