search

Franc-Z / QWen1.5_TensorRT-LLM

Optimize QWen1.5 models with TensorRT-LLM
Apache License 2.0
16 stars 3 forks source link

May I ask if these files support qwen1.5&2, can they support qwen1 at the same time? #1

Open shiqingzhangCSU opened 6 months ago

shiqingzhangCSU commented 6 months ago

As title

Franc-Z commented 6 months ago

Yes, this repo only provides solution to support QWen1.5 (by replacing Qwen1). And TensorRT-LLM has already natively support Qwen1. So if you only optimize Qwen1, this repo can't help.

shiqingzhangCSU commented 6 months ago

Yes, this repo only provides solution to support QWen1.5 (by replacing Qwen1). And TensorRT-LLM has already natively support Qwen1. So if you only optimize Qwen1, this repo can't help.

Thanks for you reply. I want use both Qwen1, QWen1.5 & 2, and I think I should add a new model type QWen2 in TensorRT-LLM( not just replacing Qwen1). Furthermore, Can it be adapted to one file like chatglm1, 2, 3, glm?

Franc-Z commented 6 months ago

For the case, you need to use both Qwen1, Qwen1.5/2, I haven't to do the solution yet. but you can do optimization sequentially.