-
This page is accessible via [roadmap.vllm.ai](https://roadmap.vllm.ai)
### Themes.
As before, we categorized our roadmap into 6 broad themes: broad model support, wide hardware coverage, state of…
-
multiple attempts to generate audio resulted in this runtime error.
relevant img attached.
followed the mamba install, using windows 10. running on rtx 3090. only change was the directory of the bar…
-
Here is the development roadmap for 2024 Q4. Contributions and feedback are welcome ([**Join Bi-weekly Development Meeting**](https://t.co/4BFjCLnVHq)). Previous 2024 Q3 roadmap can be found in #634.
…
-
# ComfyUI Error Report
## Error Details
- **Node Type:** Joy_caption_two
- **Exception Type:** ValueError
- **Exception Message:** Unrecognized model in E:\comfyui-auto\models\Joy_caption_two\te…
-
I followed the exact instructions provided by TensorRT-LLM to setup triton-llm server for whisper
I am stuck with the following error when i try to build the TRT:
```
[TensorRT-LLM] TensorRT-LLM ve…
-
Im not sure how to fix this error on windows 11:
```
Building wheel for cartopy (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for cartopy (pyproject.tom…
-
### Describe the bug
I installed text generation webui and downloaded the model(TheBloke_Yarn-Mistral-7B-128k-AWQ) and I can't run it. I chose Transofmer as Model loader. I tried installing autoawq b…
-
What should I specify as the `model_type` in the JSON file?
from transformers import AutoModel
model = AutoModel.from_pretrained("zxhezexin/openlrm-obj-base-1.1")
ValueError: Unrecogniz…
-
When I was running the 1_training_from_scratch script, there was no problem with trainer.validate(mamba_diar). However, when the trainer.fit(mamba_diar) statement was running, I encountered the follow…
-
System Info
GPU： NVIDIA RTX 4090
TensorRT-LLM 0.13
quest 1: How can I use the OpenAPI to perform inference on a TensorRT engine model?
root@docker-desktop:/llm/tensorrt-llm-0.13.0/examples/apps# pyt…