issues
search
janhq
/
cortex.tensorrt-llm
Cortex.Tensorrt-LLM is a C++ inference library that can be loaded by any server at runtime. It submodules NVIDIA’s TensorRT-LLM for GPU accelerated inference on NVIDIA's GPUs.
https://cortex.jan.ai/docs/cortex-tensorrt-llm
Apache License 2.0
37
stars
2
forks
source link
feat: Init code for cortex.tensorrtllm
#34
Closed
CameronNg
closed
4 months ago
CameronNg
commented
4 months ago
Update TRT-LLM
CMakeLists.txt
file
Setup third-party deps for
cortex.tensorrtllm
Init src code for
cortex.tensorrtllm
Prepare for server example
CMakeLists.txt
filecortex.tensorrtllm
cortex.tensorrtllm