search

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
7.73k stars 1.42k forks source link

Default Models not working #6004

Closed WaterKnight1998 closed 1 year ago

WaterKnight1998 commented 1 year ago

Description Default models from /docs/examples/model_repository are not working

Triton Information What version of Triton are you using? 2.34.0

Are you using the Triton container or did you build it yourself? Triton container: nvcr.io/nvidia/tritonserver:23.05-py3

To Reproduce docker run --gpus=1 --rm --net=host -v ${PWD}/docs/examples/model_repository:/models nvcr.io/nvidia/tritonserver:23.05-py3 tritonserver --model-repository=/models

Expected behavior All defualt models should work.

dyastremsky commented 1 year ago

Please read the README in that folder: https://github.com/triton-inference-server/server/tree/main/docs/examples. There are tutorials to follow to make use of this folder.

If you'd like to try loading them without the tutorials, you just need to run the fetch_models.sh script in that folder to get the necessary files for the densenet_onnx and inception_graphdef models. They aren't meant to be run standalone, so I'd highly recommend following the tutorial.