search

triton-inference-server / fastertransformer_backend

BSD 3-Clause "New" or "Revised" License
411 stars 134 forks source link

When hot-loading a large model, a segmentation fault will occur. #130

Open ppppppppig opened 1 year ago

ppppppppig commented 1 year ago

Description

I start triton server with '--model-control-mode poll'. Segmentation fault occurs when modifying the model directory.

Reproduced Steps

1.CUDA_VISIBLE_DEVICES=3,4,5,6 /opt/tritonserver/bin/tritonserver  --model-repository=/ft_workspace/all_models/t5/ --http-port 8008 --model-control-mode poll
2.Navigate to the directory containing the config.pbtxt file, and exec 'cp -rf 1 2'
3.client send request and segmentation fault will happen
ppppppppig commented 1 year ago

image