-
### System Info
ubuntu 20.04
tensorrt 10.0.1
tensorrt-cu12 10.0.1
tensorrt-cu12-bindings 10.0.1
tensorrt-cu12-libs 10.0.1
tensorrt-llm 0.10.…
-
The DefaultOpenAiClient in langchain4j omits the `stream` parameter in chat completion requests, even when explicitly set to `false`. This causes compatibility issues with LLM providers that interpret…
-
i saw this error :
```
value is not a valid list (type=type_error.list))
Evaluating: 33%|█████████████████████████████▎ | 1/3 [01:37 0
v…
-
## 🐛 Bug
Do not running Llama-3-8B-Instruct-q4f16_1-MLC
## To Reproduce
Steps to reproduce the behavior:
1. conda create --name mlc-prebuilt python=3.11
2. conda activate mlc-prebuilt
3…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
I'm executing following line of code:
```
new_index.storage_context.persist(pers…
-
The package complains about "torch" not being installed when it is most definitely installed.
(.env) chris@localhost:~$ pip install flash_attn
Collecting flash_attn
Using cached fla…
-
I'd like to run live llava completely locally on Jetson including a web browser.
However, if I turn off wifi before starting live llava, the video won't play on the browser.
If I turn off wifi after…
-
When I run python llava_llama_v2_visual_attack.py --n_iters 5000 --constrained --save_dir results_llava_llama_v2_constrained_16 --eps 16 --alpha 1, I meet following problems.
model = /mnt/local/LL…
-
### 🚀 The feature, motivation and pitch
Is the deepseek-v2 AWQ version supported now? When I run it, I get the following error:
```
[rank0]: File "/usr/local/lib/python3.9/dist-packages/vllm/mo…
-
I don't understand to set the chat_llm to ollama, if there is no preparation for utility_llm and/or embedding_llm to set it to local (ollama) pendants. Yes, I assume that prompting will be a challenge…