Problem
TensorRT-LLM is one of the biggest features in jan. As the docker environments usually include all kind of stuff to run the software with it's full set of features, why not include it too?
Success Criteria
A container that supports TensorRT-LLM backend out-of-the-box.
Problem TensorRT-LLM is one of the biggest features in jan. As the docker environments usually include all kind of stuff to run the software with it's full set of features, why not include it too?
Success Criteria A container that supports TensorRT-LLM backend out-of-the-box.