-
While srush has a creative way to do a roll function in Triton, I also thought I would open up an issue to see if this could potentially get implemented in the back end.
The idea would be to mimic …
-
When i train mamba2 , i firstly got a error like that:
FileNotFoundError: [Errno 2] No such file or directory: '/root/.triton/cache/2809997ca5d0d9cfa31a77d0e143bb8b/_bmm_chunk_fwd_kernel.cubin.tmp.pi…
-
### 🚀 The feature, motivation and pitch
Hi, the code can run fine. It is just that the generated comments and names are a bit confusing.
Say we have a function with some torch ops at the beginning…
-
@danielhanchen Ran into ModuleNotFoundError: No module named 'triton' while fine-tuning google/gemma-7b-it. I installed xformers successfully through a documentation that I found by Unsloth but whil…
-
**Description**
If I loaded 2 model transformer and inference model, memory GPU used about 3Gi.
```
PID USER DEV TYPE GPU GPU MEM CPU HOST MEM Command
2207044 coreai 0 C…
-
Hello, curious if we can already use sglang as a backend for NVIDIA's Triton Server.
Amazing work with the library btw, love it!
-
### System Info
Built tensorrtllm_backend from source using dockerfile/Dockerfile.trt_llm_backend
tensorrt_llm 0.13.0.dev2024081300
tritonserver 2.48.0
triton image: 24.07
Cuda 12.5
### Wh…
-
### Search before asking
- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and fou…
-
### 🐛 Describe the bug
h100-196-003:0 err: wandb: Synced 6 W&B file(s), 0 media file(s), 2 artifact file(s) and 0 other file(s)
h100-196-003:0 err: Traceback (most recent call last):
h100-196-003:0…
-
ERROR: Could not find a version that satisfies the requirement triton==2.0.0 (from versions: none)
ERROR: No matching distribution found for triton==2.0.0