Other backends are missing - Githubissues

triton-inference-server / tensorrtllm_backend

The Triton TensorRT-LLM Backend

Apache License 2.0

581 stars 81 forks source link

Other backends are missing #471

Closed Godlovecui closed 1 month ago

Godlovecui commented 1 month ago

System Info

rtx4090

Who can help?

@

Information

[ ] The official example scripts
[ ] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction

I use this docker image: nvcr.io/nvidia/tritonserver:24.04-trtllm-python-py3, however, when I run and enter container, there are two backends: why?

Expected behavior

Normal backends:

actual behavior

additional notes

None