search

NetEase-Media / grps_trtllm

【grps接入trtllm】通过接入TensorRT-LLM以及Tokenizers.cpp实现纯c++版本高性能LLM服务，兼容OpenAI接口协议，支持chat和function call模式，支持ai agent。
Apache License 2.0
46 stars 1 forks source link

Qwen2-VL Model support #1

Open atlury opened 2 weeks ago

atlury commented 2 weeks ago

Hello

Will it be possible to include support for Qwen2-VL model? Thank you

zhaocc1106 commented 2 weeks ago

Hello

Will it be possible to include support for Qwen2-VL model? Thank you

It maybe difficult now because trtllm do not support M-ROPE(https://github.com/NVIDIA/TensorRT-LLM/issues/2183). I will follow up continuously.

atlury commented 2 weeks ago

Thank you!