-
### Your current environment
The output of `python collect_env.py`
```text
$ python collect_env.py
Collecting environment information...
PyTorch version: 2.4.0+cu121
Is debug build: False
…
-
I'm trying to compile the code on my Ubuntu 18.04 LTS machine, with cuda=10.2, Nvidia driver version=440.33.01.
I suspect the code is not compatible with cuda 10.
Cuda 9 is not officially supported …
-
### Describe the issue
I have built onnxruntime-gpu 1.4.0 following . The output of `import onnxruntime` and `onnxruntime.get_device()` are both normal, and `onnxruntime.InferenceSession()` seems ok…
-
Hi, I'm using ubuntu 18.04 and 3090 with Cuda 11.1.
I encountered this problem when I tried to install it by Conda.
I already have tested Torch and Cuda, and both work.
Any advice would be thankful…
-
运行python setup.py build develop遇到错误：
fatal error: cuda_runtime_api.h: No such file or directory
#include
compilation terminated.
error: commond '/usr/bin/gcc' failed with exit code 1
-
ENV:
```
CPU: Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz
GPU: 2080Ti * 4
tensorrt-llm: 0.12.0
tensorrt: 10.3.0
```
ISSUE:
I use c++ api of "tensorrt_llm/batch_manager/" to deploy a multi-modal llm. When…
-
I've got a brand new Orin AGX developer kit (64GB) and used the SDK Manager to download and flash the latest Ubuntu (22.04) and JetPack SDK (6.1).
I'm trying to build a container (I need diffusers …
-
`cudaGraphInstantiate` signature changed in CUDA 12.
CUDA 11.8.0: https://docs.nvidia.com/cuda/archive/11.8.0/cuda-runtime-api/group__CUDART__GRAPH.html#group__CUDART__GRAPH_1gb25beab33abe4b2d13edb…
-
## Environment
- **GPUs**: 4x NVIDIA A100 (80GB) (nvlink. azure Standard_NC96ads_A100_v4)
- **TensorRT-LLM Version**: 0.15.0.dev2024102200
- **Environment**: Docker container
- **Memory Usage per GPU…
-
### 🐛 Describe the bug
Flex attention on FSDP works without compile, but not with compile. The key error seems to be `ValueError: Pointer argument (at 2) cannot be accessed from Triton (cpu tensor?)`…