-
**Submitting author:** @umutcanaltin (Umut Can Altin)
**Repository:** https://github.com/umutcanaltin/fpgai_compiler
**Branch with paper.md** (empty if default branch): main
**Version:** v1.0.0
**Edit…
-
### Is there an existing issue for this bug?
- [X] I have searched the existing issues
### 🐛 Describe the bug
Git commit: 2f583c1549(Current master branch)
## code(Example code in colossal…
-
### Search before asking
- [X] I have searched the Ultralytics YOLO [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussion…
-
**Env:**
- Container: nvcr.io/nvidia/tritonserver:23.12-trtllm-python-py3
- TensorRT-LLM release: 0.7.1
- TRT-LLM backend repo tag: v0.7.1
- Model: Llama-2-70b
- tritonserver deployed on 2 A10…
-
## Environment
- **GPUs**: 4x NVIDIA A100 (80GB) (nvlink. azure Standard_NC96ads_A100_v4)
- **TensorRT-LLM Version**: 0.15.0.dev2024102200
- **Environment**: Docker container
- **Memory Usage per GPU…
-
### Search before asking
- [X] I have searched the Ultralytics YOLO [issues](https://github.com/ultralytics/ultralytics/issues) and found no similar bug report.
### Ultralytics YOLO Component
Expo…
-
### Search before asking
- [X] I have searched the Ultralytics YOLO [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussion…
-
### Jan version
0.5.7-rc2-beta2024-11-28T03:35:22.905Z
### Describe the Bug
Basically i downloaded that model to use with Jan but when i try to activate the model got this message on the logs:
`…
-
### Name and Version
bitnami/vllm 0.1.0
### What is the problem this feature will solve?
Add the helm chart for vllm - a high-throughput and memory-efficient inference and serving engine for …
-
### Your current environment
irrelevant
### How would you like to use vllm
What would be the arguments that would maximize overall throughput for large batch offline inference? More specifically, I…