Automatic check of API backward compatibility

[Akirathan]

For the planned release on 09/2024, we should contribute some kind of automatic checking for backward compatibility of API of our libraries. Once #10369 is merged, it should not be difficult.

10369 refines the BindingsMap.exportedSymbols field to contain all the resolved symbols in the exports.

The automated backward compatibility check could work as follows:

• Store the dumped exportedSymbols field into a binary signature file for all the Main modules of our libraries.
  • Dumping can be done using our Persistance framework, that already works for that field.
• Once a change is made, compare the new exportedSymbols field with the one stored in the signature file.

What is and what is not backward compatible is yet to be defined. For example, if to_json already resolves to multiple extension methods, having even more extension methods there is backward compatible. But removing an exported symbol is not backward compatible.

Inspiration from NetBeans:

• https://github.com/jtulach/netbeans-apitest
• https://github.com/jtulach/netbeans-apitest/blob/master/src/test/resources/org/netbeans/apitest/org-netbeans-api-java.sig

Requirements

• Runs as a separate CI job. Like, for example, Code formatting check.
  • Is a separate sbt command.
• Regenerating the sigfile must not be manual.
  • signature check should not slow does the libs team.
• In case of compatibility failure, the tools shall report the exact method, its location, its old and new signatures, and the exact reason why the compatibility check failed.

[JaroslavTulach]

Prepare a design for review, Pavel.

[Akirathan]

As part of library compilation, we already generate cached Suggestions. They are later sent to the IDE in JSON format. The sigfile can thus be a JSON-formatted .suggestions. Implementation-wise this should be the most straightforward approach. Moreover, for the backward compatibility, suggestions are what we should care about the most.

[JaroslavTulach]

The sigfile can thus be a JSON-formatted .suggestions.

• Related to #9999

CCing @4e6 and @hubertp