-
### System Info
4*A800 80G
### Who can help?
@Tracin
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
- [X] An officially supported tas…
-
Microsoft recently released Phi-3 models in 3 variants (mini, small & medium). Can we add support for this new family of models.
-
**command:**
trtllm-build --checkpoint_dir gpt2/trt_ckpt/fp8/1-gpu \
--gpt_attention_plugin float16 \
--remove_input_padding enable \
--stron…
-
# clip
https://www.cnblogs.com/chester-cs/p/17478159.html
https://github.com/openai/CLIP/blob/main/clip/model.py
https://github.com/moein-shariatnia/OpenAI-CLIP/blob/master/CLIP.py
```
def …
-
### System Info
NVIDIA-SMI 535.154.05
Driver Version: 535.154.05
CUDA Version: 12.4
- GPU properties
- GPU name: NVIDIA L20
- GPU memory size: 46068MiB
- Libraries
- Te…
-
### System Info
- GPU: A100 80 GB
### Who can help?
@byshiue @kaiyux
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
- [X] An officially supported t…
-
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/batman/dev/test1/qwen_agent_env/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, i…
-
### System Info
- CPU architecture x86_64
- CPU/Host memory size 1056501432 kB
- GPU 1x A100 80GB
- [TensorRT-LLM] TensorRT-LLM version: 0.10.0.dev2024043000
- TensorRT-LLM main branch commit […
-
### System Info
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.3 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-…
-
### System Info
- CPU: x86_64
- GPU: NVIDIA GeForce RTX 2080 Ti
- Memory Size: 12 GiB
- TensorRT -LLM branch: 0.10.0.dev2024042300
- TensorRT: 9.3.0.post12.dev1
- Ubuntu: 22.04
- cuda:12.1.0
…