Use a new vllm base image with cuda 12.4

h2oai / h2ogpt

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://codellama.h2o.ai/

http://h2o.ai

Apache License 2.0

10.94k stars 1.2k forks source link

Use a new vllm base image with cuda 12.4 #1703

Closed achraf-mer closed 1 week ago