[Model Requests] Add support for CogVLM2

NVIDIA / TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

https://nvidia.github.io/TensorRT-LLM

Apache License 2.0

7.5k stars 815 forks source link

[Model Requests] Add support for CogVLM2 #1644

Open harry-stark opened 1 month ago

harry-stark commented 1 month ago

Following up on Cogvlm, CogVlm2 is here: https://github.com/THUDM/CogVLM2

Easily one of the best open-source multimodal model, that is competitive to GPT-4 and Gemini. https://github.com/THUDM/CogVLM2?tab=readme-ov-file#benchmark

The community would be grateful for having a Tensort-LLM engine for such a good model,since the support for Cogvlm #1554 has been recently added, it can be built upon it.

cc @ncomly-nvidia

isidentical commented 1 month ago

this'd be huge @AdamzNV

MultiChen commented 3 weeks ago

Very much looking forward to Tensorrt-LLM supporting CogVLM2