-
Hi, I encountered an error when trying to start the server using version 0.2.0. Specifically, I tried both the 0.2.0 package from the **releases** section with cuda 12.1 and a package I built from the…
-
### Your current environment
```text
安装完之后,直接运行python examples/minicpmv_example.py出现的问题
INFO 06-27 10:16:32 utils.py:598] Found nccl from environment variable VLLM_NCCL_SO_PATH=/usr/local/lib/pytho…
-
### What happened?
Currently on Fedora 40 with Intel Arc A750.
Running the following:
```txt
ZES_ENABLE_SYSMAN=1 ./build/bin/llama-server \
-t 10 \
-ngl 20 \
-b 512 \
--ctx-size 16384 \
-m …
-
### System Info
ubuntu 20.04
tensorrt 10.0.1
tensorrt-cu12 10.0.1
tensorrt-cu12-bindings 10.0.1
tensorrt-cu12-libs 10.0.1
tensorrt-llm 0.10.…
-
When trying to run speech. py I am getting this error:
```
speech.py", line 333, in
from TTS.tts.configs.xtts_config import XttsConfig
ModuleNotFoundError: No module named 'TTS'
```
I …
-
How do I get the input and output of each LLM call
-
### System Info
ubuntu 20.04
tensorrt 10.0.1
tensorrt-cu12 10.0.1
tensorrt-cu12-bindings 10.0.1
tensorrt-cu12-libs 10.0.1
tensorrt-llm …
-
Hi,
while trying to run this
```
python build.py --model_dir $model_dir$ \
--dtype float16 \
--use_gpt_attentionZ_plugin float16 \
--use_gemm_…
-
When I tried to add the model support for fairseq model following ops are not present,
AdaptiveSoftmax,
BaseLayer,
FairseqDropout,
LayerDropModuleList,
LayerNorm,
PositionalEmbedding,
Sinus…
-
HOST安装的步骤
conda create -n llm python=3.11
conda activate llm
# below command will install intel_extension_for_pytorch==2.1.10+xpu as default
pip install --pre --upgrade ipex-llm[xpu] --extra-index…