search

KhronosGroup / SPIRV-LLVM-Translator

A tool and a library for bi-directional translation between SPIR-V and LLVM IR
Other
477 stars 212 forks source link

SPIRV Translator releases #1898

Closed kurapov-peter closed 3 months ago

kurapov-peter commented 1 year ago

Hi folks!

We are building a library for analytical data processing that acts like a JIT. We use the translator to generate SPIRV kernels to submit to a GPU. Since this is a tool for data scientists we distribute it via their de-facto main channel - the anaconda package manager. It allows for bundling components and handling dependency versioning. Having the translator of a proper version present in their repo is thus a must for environment isolation.

Several translator versions are already present in the conda-forge channel (the recipe and build scripts can be found here: https://github.com/conda-forge/llvm-spirv-feedstock). To add a new translator version one can create a PR to the feedstock repo and update the version. The recipe is usually written in a way that requires minimal changes: only the version usually is subject to change. This way Conda can watch the new releases and update packages automatically. On update, Conda will fetch a release from the ref/tags and build a project from sources. Support for fetching a particular git revision exists though discouraged, as it results in higher resource usage compared to downloading a source's tarball.

Updates to existing branches (once LLVM version tags) would not trigger an automatic package update on the Conda side since the package version remains unchanged. Meaning, any backports will need manual work to be added. Versioning of such changes is not controlled by the translator developers in this case which is not as it should look. Any software that uses a non-release version of the component (for example, IGC pins translator version by commit SHA) faces friction in the package update process as well.

A possible solution would be to have a somewhat regular release process for the updates and backports. Package managers would be able to track the repo and update automatically saving engineering time as well as brining new functionality faster to the end user. What do you think?

bashbaug commented 1 year ago

Related issue: https://github.com/KhronosGroup/SPIRV-LLVM-Translator/issues/1508

MrSidims commented 1 year ago

Discussed today on OpenCL tooling meeting. We tend to add automatic releases for llvm_release_xx branches with a cadence of (subject of a discussion) a month. Then these releases can be used by IGC and other SPIR-V consumer to set up dependencies.

I'll take a look how to setup github actions for this.

kurapov-peter commented 1 year ago

Great news! Thanks!

ZzEeKkAa commented 4 months ago

@MrSidims any update on this?

I've raised the issue on conda forge side, but it would be handy to have pined releases https://github.com/conda-forge/llvm-spirv-feedstock/issues/23