-
### What happened?
Hi, im trying to use Google [Madlad400 in GGUF version,](https://huggingface.co/NikolayKozloff/madlad400-10b-mt-Q8_0-GGUF) but I'm unable to work it with `llama-server` but it work…
-
Hi,when I try to use command:
trtexec --onnx=decoder_model_merged.onnx --saveEngine=decoder_model_merged.trt
in linux,it showed:
[07/30/2024-02:00:24] [E] Error[4]: [graphShapeAnalyzer.cpp::analyze…
-
### System Info
Container: `nvidia/cuda:12.4.1-devel-ubuntu22.04`
GPU: L4
TensorRT-LLM version: `0.12.0.dev2024071600`
### Who can help?
_No response_
### Information
- [X] The offici…
-
### System Info / 系統信息
CogVideoX-2B SAT LoRA finetuning
### Information / 问题信息
- [ ] The official example scripts / 官方的示例脚本
- [X] My own modified scripts / 我自己修改的脚本和任务
### Reproduction / 复现过程
Fin…
-
Model loaded in 2.2s (unload existing model: 1.8s, forge model load: 0.3s).
[LORA] LoRA version mismatch for KModel: D:\StabilityMatrix-win-x64\Data\Packages\stable-diffusion-webui-forge\models\Lora\…
-
# 安装
## 开发
```shell
pip3 install ~/Ascend/ascend-toolkit/latest/lib64/te-0.4.0-py3-none-any.whl && \
pip3 install ~/Ascend/ascend-toolkit/latest/lib64/hccl-0.1.0-py3-none-any.whl && \
pip3 in…
-
Hi,
I tried to use the T5 model, as it is listed in the models package, but I get the following error:
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sentence_…
-
### Checklist
- [ ] The issue exists after disabling all extensions
- [ ] The issue exists on a clean installation of webui
- [ ] The issue is caused by an extension, but I believe it is caused by a …
-
### System Info
I am experimenting with TRT LLM and `flan-t5` models. My simple goal is to build engines with different configurations and tensor parallelism, then review performance. Have a DGX syst…
-
Hi, thanks for creating this script, amazing work! I was wondering if you have any plans in creating a convert script for T5 based models, or if you think there are any major difficulties when convert…