search

NVIDIA / TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
https://nvidia.github.io/TensorRT-LLM
Apache License 2.0
8.69k stars 994 forks source link

[Feature Request] Yi-VL support #1473

Open hitcoogle opened 7 months ago

hitcoogle commented 7 months ago

https://huggingface.co/01-ai/Yi-VL-34B

Yi Vision Language (Yi-VL) model is the open-source, multimodal version of the Yi Large Language Model (LLM) series, enabling content comprehension, recognition, and multi-round conversations about images.

Yi-VL demonstrates exceptional performance, ranking first among all existing open-source models in the latest benchmarks including MMMU in English and CMMMU in Chinese (based on data available up to January 2024).

Yi-VL-34B is the first open-source 34B vision language model worldwide.

hello-11 commented 6 days ago

cc @ncomly-nvidia @AdamzNV @laikhtewari for vis.