-
If I'm already in a docker container, how can I install TensorRT-LLM?
-
### Issue type
Bug
### Have you reproduced the bug with TensorFlow Nightly?
No
### Source
source
### TensorFlow version
2.14.0
### Custom code
Yes
### OS platform and distribution
Ubuntu 22…
-
P40下出错:
```
python3 build.py --use_weight_only --weight_only_precision=int8
[10/28/2023-03:06:29] [TRT-LLM] [I] Serially build TensorRT engines.
[10/28/2023-03:06:29] [TRT] [I] [MemUsageChange] In…
-
### Issue type
Bug
### Have you reproduced the bug with TensorFlow Nightly?
No
### Source
source
### TensorFlow version
2.14.0
### Custom code
Yes
### OS platform and distribution
Ubuntu 22…
-
### Issue type
Bug
### Have you reproduced the bug with TensorFlow Nightly?
No
### Source
source
### TensorFlow version
2.14.0
### Custom code
Yes
### OS platform and distribution
Ubuntu 22…
-
### 请提出你的问题
参考llm 的README 中llama静态图推理,报错信息如下
(paddleslim) :~/PaddleNLP/llm# python export_model.py --model_name_or_path meta-llama/Llama-2-7b-chat --output_path ./inference -
/root/miniconda3/envs/…
-
(ol) C:\Users\SNS>openllm start falcon --model-id tiiuae/falcon-7b
Downloading (…)lve/main/config.json: 100%|████████████████████████████████████████████████| 1.05k/1.05k [00:00
-
Cannot allocate memory using AWQ and Mistral based model.
```
WARNING 12-06 19:09:20 config.py:140] awq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
…
-
### Issue type
Bug
### Have you reproduced the bug with TensorFlow Nightly?
No
### Source
source
### TensorFlow version
2.14.0
### Custom code
Yes
### OS platform and distribution
Ubuntu 22…
-
### System Info
`text-generation-launcher --env
2024-02-06T18:48:34.589125Z INFO text_generation_launcher: Runtime environment:
Target: x86_64-unknown-linux-gnu
Cargo version: 1.75.0
Commit sha:…