This is a suggestion and feature discussion. Triton's issue tracker does not have a category for ideas and suggestions.
I'm from Red Hat's AI platform core components team. A few days ago I was made aware of my co-worker @maryamtahhan PR #5143 for Dev Container. Their contribution is a great hook to make my investigations public and discuss if my ideas are of interest to the Triton team. I have been working on reproducible and offline builds of Triton (e.g. #4461 and #4450). As part of the task, I looked into ways to build our own LLVM binaries for Triton. For compliance reasons, we cannot use Triton's community builds of LLVM. We have to rebuild everything from sources.
After some discussion with our internal LLVM experts and lots of experimentation, I came up with a LLVM build that is six time smaller than the current community build. For example LLVM build https://oaitriton.blob.core.windows.net/public/llvm-builds/llvm-10dc3a8e-ubuntu-x64.tar.gz is 950 MB download and almost 4 GB on disk. My build of LLVM 10dc3a8e for X86_64 is 130 MB compressed tar bundle and 650 MB uncompressed on disk. We have been using my LLVM builds for a while now and have successfully tested them with Triton 2.3.1, Triton 3.0.0, and aotriton for ROCm.
Triton's community build of LLVM is so large and slow to build, because everything is linked statically. Dynamic linking isn't an option, because it would introduce runtime dependencies on LLVM's shared libraries. My build of LLVM simply omits the majority of LLVM tools, all utils, and all docs.
I'm also using a different approach to distribute and install our LLVM builds. Instead of a tar.gz bundle, I'm using a container-first approach with an new base container that just contains the LLVM binaries. This lets us distribute the build as a multi-arch container with any OCI-conform container registry (e.g. GHCR, GitLab registry, Docker Hub, Quay). LLVM is installed into build containers with
Describe the bug
This is a suggestion and feature discussion. Triton's issue tracker does not have a category for ideas and suggestions.
I'm from Red Hat's AI platform core components team. A few days ago I was made aware of my co-worker @maryamtahhan PR #5143 for Dev Container. Their contribution is a great hook to make my investigations public and discuss if my ideas are of interest to the Triton team. I have been working on reproducible and offline builds of Triton (e.g. #4461 and #4450). As part of the task, I looked into ways to build our own LLVM binaries for Triton. For compliance reasons, we cannot use Triton's community builds of LLVM. We have to rebuild everything from sources.
After some discussion with our internal LLVM experts and lots of experimentation, I came up with a LLVM build that is six time smaller than the current community build. For example LLVM build https://oaitriton.blob.core.windows.net/public/llvm-builds/llvm-10dc3a8e-ubuntu-x64.tar.gz is 950 MB download and almost 4 GB on disk. My build of LLVM
10dc3a8efor X86_64 is 130 MB compressed tar bundle and 650 MB uncompressed on disk. We have been using my LLVM builds for a while now and have successfully tested them with Triton 2.3.1, Triton 3.0.0, and aotriton for ROCm.Triton's community build of LLVM is so large and slow to build, because everything is linked statically. Dynamic linking isn't an option, because it would introduce runtime dependencies on LLVM's shared libraries. My build of LLVM simply omits the majority of LLVM tools, all utils, and all docs.
I'm also using a different approach to distribute and install our LLVM builds. Instead of a
tar.gzbundle, I'm using a container-first approach with an new base container that just contains the LLVM binaries. This lets us distribute the build as a multi-arch container with any OCI-conform container registry (e.g. GHCR, GitLab registry, Docker Hub, Quay). LLVM is installed into build containers withIt takes mere seconds to copy a layer from one container to another. The container-first approach doesn't work for everybody, though.
Would you be interested to include some of the LLVM changes into Triton?
Environment details
LLVM for Linux