SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.
Triton Kernel Fix:
If you see No such file or directory: '/root/.triton/cache/e3457c918521f16104a655b081235e5a.....
(issue caused by pytorch dependency of triton==2.3.0)
You can fix it by hacking the file compiler.py
vim /usr/local/lib/python3.10/dist-packages/triton/compiler/compiler.py
Triton Kernel Fix: If you see
No such file or directory: '/root/.triton/cache/e3457c918521f16104a655b081235e5a.....(issue caused by pytorch dependency of triton==2.3.0)compiler.pyL230
self.asm = { file.suffix[1:]: file.read_bytes() if file.suffix[1:] == driver.binary_ext else None
pip install -U --index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/Triton-Nightly/pypi/simple/ triton-nightly