Closed JamesBowerXanda closed 1 week ago
Hi @JamesBowerXanda, Triton doesn't officially support Mac, but I assume it would work if you are only running CPU-only model. I couldn't reproduce the hang using a linux machine. Since I don't have the wav2vec2_asr_base_960h.pt
and labels.json
files, I replaced the wav2vec2_asr_base_960h.pt
with some model.pt and remove the line for labels.json
, Triton is not hanging on my side. Could you run the server with --log-verbose=1
and see if there's any error reported in the log?
I also notice that the path for those two files might be incorrect
model_repository = args["model_repository"]
wav2vec2_path = os.path.join(model_repository,"1","wav2vec2_asr_base_960h.pt")
labels_path = os.path.join(model_repository,"1","labels.json")
The args["model_repository"]
will return model_repository/forced_alignmen
while the wav2vec2_asr_base_960h.pt
and labels.json
files are under model_repository/baai_quant_onnx/1/
.
Closing due to lack of activity. Please re-open the issue if you would like to follow up with this issue.
Description I am trying to use the triton server on cpu only model and during launch the server will launch perfectly with only ONNX models but the moment I include a python backend model it hangs on launch eternally.
I am using an Apple M2 Mac.
It is worth noting that the model runs when I use the Sagemaker Triton Server Image on a sagemaker multimodel endpoint.
Triton Information
Version? 23.02 although have also tried 24.04.
Are you using the Triton container or did you build it yourself? Container. Specifically
nvcr.io/nvidia/tritonserver:23.02-py3
.To Reproduce
docker run -it -p8000:8000 -p8001:8001 -p8002:8002 -v/Users/jamesbower/Projects/triton-local/model_repository:/models nvcr.io/nvidia/tritonserver:23.02-py3 /bin/bash
pip install --no-cache-dir torch --index-url https://download.pytorch.org/whl/cpu
andpip install --no-cache-dir numpy
.tritonserver --model-repository models/
Output
The following is displayed:
It just hangs here eternally.
Expected behavior Triton server launched completely such that
curl -v localhost:8000/v2/health/ready
receives a status 200 response.Model Repository Setup
The structure of the model repository is:
I am not using a conda packed execution environment since I install the required packages in the container after launching it. I have also tried with a conda packed conda env though which is the method I used with the SageMaker Triton Server image.
The model.py file is
The config.pbtxt is:
Execution env is not set as I install the required packages in the container.