search

NVIDIA / TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
https://nvidia.github.io/TensorRT-LLM
Apache License 2.0
8.6k stars 978 forks source link

Add support for Nougat model #2098

Closed sherlcok314159 closed 2 months ago

sherlcok314159 commented 3 months ago

System Info

...

Who can help?

No response

Information

Tasks

Reproduction

The nougat source code is here: https://github.com/facebookresearch/nougat.

Expected behavior

Hope we can use TensorRTLLM with Nougat.

actual behavior

Nougat is not in support.

additional notes

Thanks for your contribution!

lfr-0531 commented 2 months ago

TensorRT-LLM can support Nougat model, please refer to the multimodal examples: https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/multimodal#nougat