[RFC] New versioning scheme for IREE Python packages

[marbre]

Summary

Our pre-compiled software is currently made publicly available using different versioning schemes. We're proposing a new versioning scheme to remedy this situation to which downstream packages depending on IREE can easily adopt.

This affects these packages:

• https://pypi.org/project/iree-compiler/ (current version 20240828.999)
• https://pypi.org/project/iree-runtime/ (current version 20240828.999)
• https://pypi.org/project/iree-turbine/ (current version 2.4.1)
• (maybe) https://pypi.org/project/iree-tools-tflite/ (current version 20240226.813)
• (maybe) https://pypi.org/project/iree-tools-tf/ (current version 20240226.813)

Here are examples of the new versions we propose:

Type of build	Version format	Version example
Stable release (PyPI)	X.Y.Z	3.0.0
Nightly release (GitHub schedule)	X.Y.ZrcYYYYMMDD	3.0.1rc20241029
Dev release (GitHub pull_request)	X.Y.Z.devNN	3.0.1.dev50
Local build	X.Y.Z.dev	3.0.1.dev

Key

Identifier	Explanation
X	Major version
Y	Minor version
Z	Patch version
rc	release candidate (main branch)
dev	developer build (code on pull request branches)
YYYY	Year, e.g. 2024
MM	Month, e.g. 10
DD	Day, e.g. 29
NN	Build number (increasing ID from GitHub per workflow), e.g. 50

Background

Currently CalVer is used for releasing iree-compiler and iree-runtime whereas iree-turbine utilizes a version number in the format of X.Y.Z. Therefore, compatible versions cannot be derived from the version number. A common versioning scheme would be SemVer, but as the MAJOR version must be increased when incompatible API changes are introduced a cross-project version number (primarily a shared MAJOR version) cannot be guaranteed if APIs are not identical. Switching to a non-CalVer scheme requires a package rename as the first release segment would otherwise need to be set to a number larger than 20240928, e.g. 30000000, or to introduce an N! epoch segment which however is not a convenient solution, see Python version epochs are broken.

Detailed proposal

A shared version format is proposed for the packages

• iree-base-compiler (formally named iree-compiler)
• iree-base-runtime (formally named iree-runtime)
• iree-turbine

Major or minor releases are published every 6 weeks for all packages uniformly. Based on logistics and/or practical engineering issues, releases may be delayed or deferred as needed. Patch versions for individual packages can be published outside of that cycle as needed.

A release number is in the format of X.Y.Z (MAJOR.MAJOR.PATCH).

• X and Y are defined as shared version numbers between all packages.
• The patch level Z MAY be incremented individually.
• A PATCH release contains only bug fixes and the version Z (x.y.Z) MUST be incremented. A bug fix is an internal change that fixes incorrect behavior and MUST NOT introduce breaking changes.
• A MINOR release (unlike SemVer) as well as a MAJOR release MAY contain backwards-incompatible, breaking changes, like API changes and removals and furthermore bug fixes and new features.

Development and nightly releases:

• Development builds (e.g. from a regular CI) MUST be released with a version number defined as X.Y.Z.devN, where N is a build number.
• Nightly releases MUST be released with a version number defined as X.Y.ZrcYYYYMMDD.
• The intent is to promote a recent, high quality release candidate to a final version. Binary stamps and tools will continue to report the original release candidate version.

The following semantics apply:

• If the version X (X.y.z) is increased for one package, the version number change MUST be adopted by all (other) packages. The same applies for the version Y (x.Y.Z).
• If the version X or Y are changed, Z MUST be set 0.
• After a release, either Y or Z MUST be increased to ensure precedence of nightly builds. For example:
  • The latest stable release published on November 15th 2024 is versioned as version 3.0.0.
  • The next nightly builds are released as 3.0.1rc20241116 or 3.1.0rc20241116.
  • The next stable release is released as 3.0.1 or 3.1.0.

Additional version information

With switching from CalVer to the proposed versioning scheme, the build date is not encoded in the version number any longer. However, additional information is provided by --version, e.g. iree-compile --version gives:

IREE (https://iree.dev):
  IREE compiler version 20240730.970 @ d39c3c56682e006e842b32aa6f38c272f77c8f3c
  LLVM version 20.0.0git
  Optimized build

which can be extended by the build date.

Implementation plan

Each project will have a version_info.json file containing the latest X.Y.Z version. Local, pull request, and nightly builds will read from this file and append their own suffixes to it. When a new release is promoted to PyPI from a nightly release, a script will be used to edit the embedded version, removing any rcYYYYMMDD suffix and then we will upgrade the version in each version_info.json file so future builds will use new versions. The new releases will start at version 3.0.0, since iree-turbine is currently 2.4.1.

Timeline

We would like to begin implementation work soon and have this completed within ~2 weeks.

[ScottTodd]

Highlighting an important detail that could be missed: with these version changes, existing workflows that use

python -m pip install iree-compiler iree-runtime

will need to be replaced with new packages

python -m pip install iree-base-compiler iree-base-runtime

Simply switching the version scheme on the existing packages would require starting with a version greater than the existing calver (20240828), or using Python version epochs, which are broken.

After rolling out the initial releases for each package, we can also explore ergonomics improvements like functional or meta packages that themselves depend on both "base" packages, so users can install a single project or the individual packages if they want precise control.

We can also publish tombstone package releases to the iree-compiler and iree-runtime projects that warn about the name change and maybe eventually (after years?) remove the old names/packages entirely.