Closed ghost closed 3 years ago
Tabrizian
commented
3 years ago @ihcho9088 Did you compile the Python backend stub as well? The version of Python that is used in your conda environment is slightly different than the version used by Python backend. (3.8.10 vs 3.8.11). https://github.com/triton-inference-server/python_backend#1-building-custom-python-backend-stub
ghost
commented
3 years ago @Tabrizian
I have created stub but it complained as well.
Here is a short version of the full log. It used the environment archive file test2.tar.gz (python version 3.8.11) and the stub compiled by the user.
I0804 00:10:22.183898 1 python.cc:1275] Using Python execution env /models/wav2lip/test2.tar.gz
I0804 00:10:22.185185 1 python.cc:1497] TRITONBACKEND_ModelInstanceInitialize: wav2lip_0 (GPU device 0)
I0804 00:10:22.186486 1 backend_model_instance.cc:105] Creating instance wav2lip_0 on GPU 0 (7.0) using artifact ''
I0804 00:10:29.322827 56 python.cc:918] Starting Python backend stub: export LD_LIBRARY_PATH=/tmp/python_env_y6A8Nf/0/lib:$LD_LIBRARY_PATH; source /tmp/python_env_y6A8Nf/0/bin/activate && exec /models/wav2lip/triton_python_backend_stub /models/wav2lip/1/model.py /wav2lip_0_GPU_0 67108864 67108864 1 /opt/tritonserver/backends/python
0804 00:10:29.919358 56 pb_stub.cc:639] ModuleNotFoundError: No module named 'decorator'The strange thing is that the same error was raised with python version 3.8.10 without a custom compiled stub.
Here is a short version of the full log. It used the environment archive file test3.tar.gz (python version 3.8.10) and off-the-shelf stub.
I0804 00:20:38.780090 1 python.cc:1275] Using Python execution env /models/wav2lip/test3.tar.gz
I0804 00:20:38.781313 1 python.cc:1497] TRITONBACKEND_ModelInstanceInitialize: wav2lip_0 (GPU device 0)
I0804 00:20:38.782551 1 backend_model_instance.cc:105] Creating instance wav2lip_0 on GPU 0 (7.0) using artifact ''
I0804 00:20:46.091840 56 python.cc:918] Starting Python backend stub: export LD_LIBRARY_PATH=/tmp/python_env_zQcmpg/0/lib:$LD_LIBRARY_PATH; source /tmp/python_env_zQcmpg/0/bin/activate && exec /opt/tritonserver/backends/python/triton_python_backend_stub /models/wav2lip/1/model.py /wav2lip_0_GPU_0 67108864 67108864 1 /opt/tritonserver/backends/python
0804 00:20:46.782380 56 pb_stub.cc:639] ModuleNotFoundError: No module named 'decorator'
$ conda activate test2
$ git clone https://github.com/triton-inference-server/python_backend -b r21.07
$ mkdir -p python_backend/build && cd python_backend/build
$ cmake -DTRITON_ENABLE_GPU=ON -DCMAKE_INSTALL_PREFIX:PATH=`pwd`/install ..
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- RapidJSON found. Headers: /usr/include
-- RapidJSON found. Headers: /usr/include
-- Found CUDAToolkit: /usr/local/cuda/include (found version "11.0.194")
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda (found version "11.0")
-- Using CUDA 11.0
-- pybind11 v2.6.2
-- Found PythonInterp: /home/andy/miniconda3/envs/test2/bin/python (found version "3.8.11")
-- Found PythonLibs: /home/andy/miniconda3/envs/test2/lib/libpython3.8.so
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/andy/triton/python_backend/build$ make triton-python-backend-stub [ 4%] Building NVCC (Device) object _deps/repo-backend-build/CMakeFiles/kernel-library-new.dir/src/kernel-library-new_generated_kernel.cu.o [ 8%] Linking CXX static library libkernel-library-new.a [ 8%] Built target kernel-library-new [ 12%] Creating directories for 'boostorg' [ 16%] Performing download step (download, verify and extract) for 'boostorg' -- Downloading... dst='/home/andy/triton/python_backend/build/boost-src/src/boost_1_76_0.tar.gz' timeout='none' inactivity timeout='none' -- Using src='https://boostorg.jfrog.io/artifactory/main/release/1.76.0/source/boost_1_76_0.tar.gz' -- [download 0% complete] -- [download 1% complete] -- [download 2% complete] -- [download 3% complete] -- [download 4% complete] -- [download 5% complete] -- [download 6% complete] -- [download 7% complete] -- [download 8% complete] -- [download 9% complete] -- [download 10% complete] -- [download 11% complete] -- [download 12% complete] -- [download 13% complete] -- [download 14% complete] -- [download 15% complete] -- [download 16% complete] -- [download 17% complete] -- [download 18% complete] -- [download 19% complete] -- [download 20% complete] -- [download 21% complete] -- [download 22% complete] -- [download 23% complete] -- [download 24% complete] -- [download 25% complete] -- [download 26% complete] -- [download 27% complete] -- [download 28% complete] -- [download 29% complete] -- [download 30% complete] -- [download 31% complete] -- [download 32% complete] -- [download 33% complete] -- [download 34% complete] -- [download 35% complete] -- [download 36% complete] -- [download 37% complete] -- [download 38% complete] -- [download 39% complete] -- [download 40% complete] -- [download 41% complete] -- [download 42% complete] -- [download 43% complete] -- [download 44% complete] -- [download 45% complete] -- [download 46% complete] -- [download 47% complete] -- [download 48% complete] -- [download 49% complete] -- [download 50% complete] -- [download 51% complete] -- [download 52% complete] -- [download 53% complete] -- [download 54% complete] -- [download 55% complete] -- [download 56% complete] -- [download 57% complete] -- [download 58% complete] -- [download 59% complete] -- [download 60% complete] -- [download 61% complete] -- [download 62% complete] -- [download 63% complete] -- [download 64% complete] -- [download 65% complete] -- [download 66% complete] -- [download 67% complete] -- [download 68% complete] -- [download 69% complete] -- [download 70% complete] -- [download 71% complete] -- [download 72% complete] -- [download 73% complete] -- [download 74% complete] -- [download 75% complete] -- [download 76% complete] -- [download 77% complete] -- [download 78% complete] -- [download 79% complete] -- [download 80% complete] -- [download 81% complete] -- [download 82% complete] -- [download 83% complete] -- [download 84% complete] -- [download 85% complete] -- [download 86% complete] -- [download 87% complete] -- [download 88% complete] -- [download 89% complete] -- [download 90% complete] -- [download 91% complete] -- [download 92% complete] -- [download 93% complete] -- [download 94% complete] -- [download 95% complete] -- [download 96% complete] -- [download 97% complete] -- [download 98% complete] -- [download 99% complete] -- [download 100% complete] -- verifying file... file='/home/andy/triton/python_backend/build/boost-src/src/boost_1_76_0.tar.gz' -- Downloading... done -- extracting... src='/home/andy/triton/python_backend/build/boost-src/src/boost_1_76_0.tar.gz' dst='/home/andy/triton/python_backend/build/boost-src/src/boostorg' -- extracting... [tar xfz] -- extracting... [analysis] -- extracting... [rename] -- extracting... [clean up] -- extracting... done [ 20%] No update step for 'boostorg' [ 25%] No patch step for 'boostorg' [ 29%] Performing configure step for 'boostorg' [ 33%] No build step for 'boostorg' [ 37%] No install step for 'boostorg' [ 41%] Completed 'boostorg' [ 41%] Built target boostorg [ 45%] Building CXX object _deps/repo-common-build/CMakeFiles/triton-common-async-work-queue.dir/src/async_work_queue.cc.o [ 50%] Building CXX object _deps/repo-common-build/CMakeFiles/triton-common-async-work-queue.dir/src/error.cc.o [ 54%] Linking CXX static library libtritonasyncworkqueue.a [ 54%] Built target triton-common-async-work-queue [ 58%] Building CXX object _deps/repo-backend-build/CMakeFiles/triton-backend-utils.dir/src/backend_common.cc.o [ 62%] Building CXX object _deps/repo-backend-build/CMakeFiles/triton-backend-utils.dir/src/backend_input_collector.cc.o [ 66%] Building CXX object _deps/repo-backend-build/CMakeFiles/triton-backend-utils.dir/src/backend_memory.cc.o [ 70%] Building CXX object _deps/repo-backend-build/CMakeFiles/triton-backend-utils.dir/src/backend_model_instance.cc.o [ 75%] Building CXX object _deps/repo-backend-build/CMakeFiles/triton-backend-utils.dir/src/backend_model.cc.o [ 79%] Building CXX object _deps/repo-backend-build/CMakeFiles/triton-backend-utils.dir/src/backend_output_responder.cc.o [ 83%] Linking CXX static library libtritonbackendutils.a [ 83%] Built target triton-backend-utils [ 87%] Building CXX object CMakeFiles/triton-python-backend-stub.dir/src/pb_stub.cc.o [ 91%] Building CXX object CMakeFiles/triton-python-backend-stub.dir/src/pb_utils.cc.o [ 95%] Building CXX object CMakeFiles/triton-python-backend-stub.dir/src/shm_manager.cc.o [100%] Linking CXX executable triton_python_backend_stub [100%] Built target triton-python-backend-stub
$ ldd triton_python_backend_stub linux-vdso.so.1 (0x00007fff9cbc3000) libarchive.so.13 => /usr/lib/x86_64-linux-gnu/libarchive.so.13 (0x00007efc44c5d000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007efc44a55000) libpython3.8.so.1.0 => /home/user/miniconda3/envs/test2/lib/libpython3.8.so.1.0 (0x00007efc446b3000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007efc44494000) libstdc++.so.6 => /home/user/miniconda3/envs/test2/lib/libstdc++.so.6 (0x00007efc45219000) libgcc_s.so.1 => /home/user/miniconda3/envs/test2/lib/libgcc_s.so.1 (0x00007efc45205000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007efc440a3000) libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007efc43e6d000) libacl.so.1 => /lib/x86_64-linux-gnu/libacl.so.1 (0x00007efc43c65000) liblzo2.so.2 => /lib/x86_64-linux-gnu/liblzo2.so.2 (0x00007efc43a43000) liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007efc4381d000) liblz4.so.1 => /usr/lib/x86_64-linux-gnu/liblz4.so.1 (0x00007efc43601000) libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007efc433f1000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007efc431d4000) libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007efc42e13000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007efc42c0f000) libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007efc42a0c000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007efc4266e000) /lib64/ld-linux-x86-64.so.2 (0x00007efc45177000) libattr.so.1 => /lib/x86_64-linux-gnu/libattr.so.1 (0x00007efc42469000) libicuuc.so.60 => /usr/lib/x86_64-linux-gnu/libicuuc.so.60 (0x00007efc420b1000) libicudata.so.60 => /usr/lib/x86_64-linux-gnu/libicudata.so.60 (0x00007efc40508000)
$ mv triton_python_backend_stub /path/to/model_repository/wav2lip
2. Output
```console
$ docker run --gpus=1 --shm-size=5G -p8000:8000 -p8001:8001 -p8002:8002 --rm -v /path/to/model_repository:/models --name tritonserver nvcr.io/nvidia/tritonserver:21.07-py3 tritonserver --model-repository=/models --log-verbose 20
=============================
== Triton Inference Server ==
=============================
NVIDIA Release 21.07 (build 24810355)
Copyright (c) 2018-2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
Various files include modifications (c) NVIDIA CORPORATION. All rights reserved.
This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
NOTE: Legacy NVIDIA Driver detected. Compatibility mode ENABLED.
I0804 00:10:21.169006 1 metrics.cc:290] Collecting metrics for GPU 0: Tesla V100-PCIE-16GB
I0804 00:10:21.169397 1 shared_library.cc:108] OpenLibraryHandle: /opt/tritonserver/backends/pytorch/libtriton_pytorch.so
I0804 00:10:21.465437 1 libtorch.cc:998] TRITONBACKEND_Initialize: pytorch
I0804 00:10:21.465489 1 libtorch.cc:1008] Triton TRITONBACKEND API version: 1.4
I0804 00:10:21.465495 1 libtorch.cc:1014] 'pytorch' TRITONBACKEND API version: 1.4
I0804 00:10:21.465564 1 shared_library.cc:108] OpenLibraryHandle: /opt/tritonserver/backends/tensorflow1/libtriton_tensorflow1.so
2021-08-04 00:10:21.617223: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
I0804 00:10:21.660712 1 tensorflow.cc:2169] TRITONBACKEND_Initialize: tensorflow
I0804 00:10:21.660748 1 tensorflow.cc:2179] Triton TRITONBACKEND API version: 1.4
I0804 00:10:21.660754 1 tensorflow.cc:2185] 'tensorflow' TRITONBACKEND API version: 1.4
I0804 00:10:21.660759 1 tensorflow.cc:2209] backend configuration:
{}
I0804 00:10:21.660816 1 shared_library.cc:108] OpenLibraryHandle: /opt/tritonserver/backends/onnxruntime/libtriton_onnxruntime.so
I0804 00:10:21.662185 1 onnxruntime.cc:1970] TRITONBACKEND_Initialize: onnxruntime
I0804 00:10:21.662210 1 onnxruntime.cc:1980] Triton TRITONBACKEND API version: 1.4
I0804 00:10:21.662216 1 onnxruntime.cc:1986] 'onnxruntime' TRITONBACKEND API version: 1.4
I0804 00:10:21.671951 1 shared_library.cc:108] OpenLibraryHandle: /opt/tritonserver/backends/openvino/libtriton_openvino.so
I0804 00:10:21.684076 1 openvino.cc:1193] TRITONBACKEND_Initialize: openvino
I0804 00:10:21.684100 1 openvino.cc:1203] Triton TRITONBACKEND API version: 1.4
I0804 00:10:21.684105 1 openvino.cc:1209] 'openvino' TRITONBACKEND API version: 1.4
I0804 00:10:22.076096 1 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7f2668000000' with size 268435456
I0804 00:10:22.076504 1 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864
I0804 00:10:22.077106 1 backend_factory.h:45] Create TritonBackendFactory
I0804 00:10:22.077131 1 plan_backend_factory.cc:49] Create PlanBackendFactory
I0804 00:10:22.077136 1 plan_backend_factory.cc:56] Registering TensorRT Plugins
I0804 00:10:22.077193 1 logging.cc:52] Registered plugin creator - ::GridAnchor_TRT version 1
I0804 00:10:22.077213 1 logging.cc:52] Registered plugin creator - ::GridAnchorRect_TRT version 1
I0804 00:10:22.077233 1 logging.cc:52] Registered plugin creator - ::NMS_TRT version 1
I0804 00:10:22.077246 1 logging.cc:52] Registered plugin creator - ::Reorg_TRT version 1
I0804 00:10:22.077255 1 logging.cc:52] Registered plugin creator - ::Region_TRT version 1
I0804 00:10:22.077272 1 logging.cc:52] Registered plugin creator - ::Clip_TRT version 1
I0804 00:10:22.077281 1 logging.cc:52] Registered plugin creator - ::LReLU_TRT version 1
I0804 00:10:22.077294 1 logging.cc:52] Registered plugin creator - ::PriorBox_TRT version 1
I0804 00:10:22.077303 1 logging.cc:52] Registered plugin creator - ::Normalize_TRT version 1
I0804 00:10:22.077312 1 logging.cc:52] Registered plugin creator - ::ScatterND version 1
I0804 00:10:22.077321 1 logging.cc:52] Registered plugin creator - ::RPROI_TRT version 1
I0804 00:10:22.077333 1 logging.cc:52] Registered plugin creator - ::BatchedNMS_TRT version 1
I0804 00:10:22.077344 1 logging.cc:52] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
I0804 00:10:22.077352 1 logging.cc:52] Registered plugin creator - ::FlattenConcat_TRT version 1
I0804 00:10:22.077371 1 logging.cc:52] Registered plugin creator - ::CropAndResize version 1
I0804 00:10:22.077385 1 logging.cc:52] Registered plugin creator - ::DetectionLayer_TRT version 1
I0804 00:10:22.077394 1 logging.cc:52] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
I0804 00:10:22.077404 1 logging.cc:52] Registered plugin creator - ::EfficientNMS_TRT version 1
I0804 00:10:22.077414 1 logging.cc:52] Registered plugin creator - ::Proposal version 1
I0804 00:10:22.077422 1 logging.cc:52] Registered plugin creator - ::ProposalLayer_TRT version 1
I0804 00:10:22.077434 1 logging.cc:52] Registered plugin creator - ::PyramidROIAlign_TRT version 1
I0804 00:10:22.077442 1 logging.cc:52] Registered plugin creator - ::ResizeNearest_TRT version 1
I0804 00:10:22.077456 1 logging.cc:52] Registered plugin creator - ::Split version 1
I0804 00:10:22.077482 1 logging.cc:52] Registered plugin creator - ::SpecialSlice_TRT version 1
I0804 00:10:22.077501 1 logging.cc:52] Registered plugin creator - ::InstanceNormalization_TRT version 1
I0804 00:10:22.077513 1 ensemble_backend_factory.cc:47] Create EnsembleBackendFactory
I0804 00:10:22.079593 1 model_repository_manager.cc:749] AsyncLoad() 'wav2lip'
I0804 00:10:22.079665 1 model_repository_manager.cc:988] TriggerNextAction() 'wav2lip' version 1: 1
I0804 00:10:22.079680 1 model_repository_manager.cc:1026] Load() 'wav2lip' version 1
I0804 00:10:22.079685 1 model_repository_manager.cc:1045] loading: wav2lip:1
I0804 00:10:22.179942 1 model_repository_manager.cc:1105] CreateInferenceBackend() 'wav2lip' version 1
I0804 00:10:22.180070 1 shared_library.cc:108] OpenLibraryHandle: /opt/tritonserver/backends/python/libtriton_python.so
I0804 00:10:22.182209 1 python.cc:1306] 'python' TRITONBACKEND API version: 1.4
I0804 00:10:22.182228 1 python.cc:1328] backend configuration:
{}
I0804 00:10:22.182239 1 python.cc:1405] shm-default-byte-size=67108864,shm-growth-byte-size=67108864,stub-timeout-seconds=30
I0804 00:10:22.182604 1 python.cc:1453] TRITONBACKEND_ModelInitialize: wav2lip (version 1)
I0804 00:10:22.183712 1 model_config_utils.cc:1521] ModelConfig 64-bit fields:
I0804 00:10:22.183733 1 model_config_utils.cc:1523] ModelConfig::dynamic_batching::default_queue_policy::default_timeout_microseconds
I0804 00:10:22.183738 1 model_config_utils.cc:1523] ModelConfig::dynamic_batching::max_queue_delay_microseconds
I0804 00:10:22.183742 1 model_config_utils.cc:1523] ModelConfig::dynamic_batching::priority_queue_policy::value::default_timeout_microseconds
I0804 00:10:22.183747 1 model_config_utils.cc:1523] ModelConfig::ensemble_scheduling::step::model_version
I0804 00:10:22.183751 1 model_config_utils.cc:1523] ModelConfig::input::dims
I0804 00:10:22.183754 1 model_config_utils.cc:1523] ModelConfig::input::reshape::shape
I0804 00:10:22.183758 1 model_config_utils.cc:1523] ModelConfig::instance_group::secondary_devices::device_id
I0804 00:10:22.183763 1 model_config_utils.cc:1523] ModelConfig::model_warmup::inputs::value::dims
I0804 00:10:22.183766 1 model_config_utils.cc:1523] ModelConfig::optimization::cuda::graph_spec::graph_lower_bound::input::value::dim
I0804 00:10:22.183771 1 model_config_utils.cc:1523] ModelConfig::optimization::cuda::graph_spec::input::value::dim
I0804 00:10:22.183775 1 model_config_utils.cc:1523] ModelConfig::output::dims
I0804 00:10:22.183779 1 model_config_utils.cc:1523] ModelConfig::output::reshape::shape
I0804 00:10:22.183783 1 model_config_utils.cc:1523] ModelConfig::sequence_batching::direct::max_queue_delay_microseconds
I0804 00:10:22.183787 1 model_config_utils.cc:1523] ModelConfig::sequence_batching::max_sequence_idle_microseconds
I0804 00:10:22.183791 1 model_config_utils.cc:1523] ModelConfig::sequence_batching::oldest::max_queue_delay_microseconds
I0804 00:10:22.183795 1 model_config_utils.cc:1523] ModelConfig::version_policy::specific::versions
I0804 00:10:22.183898 1 python.cc:1275] Using Python execution env /models/wav2lip/test2.tar.gz
I0804 00:10:22.185185 1 python.cc:1497] TRITONBACKEND_ModelInstanceInitialize: wav2lip_0 (GPU device 0)
I0804 00:10:22.186486 1 backend_model_instance.cc:105] Creating instance wav2lip_0 on GPU 0 (7.0) using artifact ''
I0804 00:10:29.322827 56 python.cc:918] Starting Python backend stub: export LD_LIBRARY_PATH=/tmp/python_env_y6A8Nf/0/lib:$LD_LIBRARY_PATH; source /tmp/python_env_y6A8Nf/0/bin/activate && exec /models/wav2lip/triton_python_backend_stub /models/wav2lip/1/model.py /wav2lip_0_GPU_0 67108864 67108864 1 /opt/tritonserver/backends/python
0804 00:10:29.919358 56 pb_stub.cc:639] ModuleNotFoundError: No module named 'decorator'
At:
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/_cache.py(40): wrapper
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/util/utils.py(230): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/util/__init__.py(83): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap>(961): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/core/notation.py(8): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap>(1050): _handle_fromlist
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/core/convert.py(7): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/core/__init__.py(5): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap>(1050): _handle_fromlist
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/__init__.py(211): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/models/wav2lip/1/model.py(13): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
I0804 00:10:29.919543 1 python.cc:1557] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0804 00:10:29.919763 1 python.cc:1476] TRITONBACKEND_ModelFinalize: delete model state
I0804 00:10:29.919798 1 triton_backend_manager.cc:101] unloading backend 'python'
I0804 00:10:29.919803 1 python.cc:1433] TRITONBACKEND_Finalize: Start
I0804 00:10:30.277683 1 python.cc:1438] TRITONBACKEND_Finalize: End
E0804 00:10:30.280266 1 model_repository_manager.cc:1215] failed to load 'wav2lip' version 1: Internal: ModuleNotFoundError: No module named 'decorator'
At:
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/_cache.py(40): wrapper
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/util/utils.py(230): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/util/__init__.py(83): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap>(961): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/core/notation.py(8): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap>(1050): _handle_fromlist
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/core/convert.py(7): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/core/__init__.py(5): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap>(1050): _handle_fromlist
/tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/__init__.py(211): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/models/wav2lip/1/model.py(13): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(843): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
I0804 00:10:30.280305 1 model_repository_manager.cc:988] TriggerNextAction() 'wav2lip' version 1: 0
I0804 00:10:30.280317 1 model_repository_manager.cc:1003] no next action, trigger OnComplete()
I0804 00:10:30.280401 1 model_repository_manager.cc:594] VersionStates() 'wav2lip'
I0804 00:10:30.280465 1 model_repository_manager.cc:594] VersionStates() 'wav2lip'
I0804 00:10:30.280523 1 server.cc:504]
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+
I0804 00:10:30.280607 1 server.cc:543]
+-------------+-----------------------------------------------------------------+--------+
| Backend | Path | Config |
+-------------+-----------------------------------------------------------------+--------+
| tensorrt | <built-in> | {} |
| pytorch | /opt/tritonserver/backends/pytorch/libtriton_pytorch.so | {} |
| tensorflow | /opt/tritonserver/backends/tensorflow1/libtriton_tensorflow1.so | {} |
| onnxruntime | /opt/tritonserver/backends/onnxruntime/libtriton_onnxruntime.so | {} |
| openvino | /opt/tritonserver/backends/openvino/libtriton_openvino.so | {} |
+-------------+-----------------------------------------------------------------+--------+
I0804 00:10:30.280620 1 model_repository_manager.cc:570] BackendStates()
I0804 00:10:30.280725 1 server.cc:586]
+---------+---------+-----------------------------------------------------------------------------------------------+
| Model | Version | Status |
+---------+---------+-----------------------------------------------------------------------------------------------+
| wav2lip | 1 | UNAVAILABLE: Internal: ModuleNotFoundError: No module named 'decorator' |
| | | |
| | | At: |
| | | /tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/_cache.py(40): wrapper |
| | | /tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/util/utils.py(230): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(843): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | /tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/util/__init__.py(83): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(843): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap>(961): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | /tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/core/notation.py(8): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(843): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap>(1050): _handle_fromlist |
| | | /tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/core/convert.py(7): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(843): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | /tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/core/__init__.py(5): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(843): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap>(1050): _handle_fromlist |
| | | /tmp/python_env_y6A8Nf/0/lib/python3.8/site-packages/librosa/__init__.py(211): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(843): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | /models/wav2lip/1/model.py(13): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(843): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
+---------+---------+-----------------------------------------------------------------------------------------------+
I0804 00:10:30.280839 1 tritonserver.cc:1718]
+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Option | Value |
+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| server_id | triton |
| server_version | 2.12.0 |
| server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data statistics |
| model_repository_path[0] | /models |
| model_control_mode | MODE_NONE |
| strict_model_config | 1 |
| pinned_memory_pool_byte_size | 268435456 |
| cuda_memory_pool_byte_size{0} | 67108864 |
| min_supported_compute_capability | 6.0 |
| strict_readiness | 1 |
| exit_timeout | 30 |
+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I0804 00:10:30.280851 1 server.cc:234] Waiting for in-flight requests to complete.
I0804 00:10:30.280855 1 model_repository_manager.cc:694] AsyncUnload() 'wav2lip'
I0804 00:10:30.280860 1 model_repository_manager.cc:988] TriggerNextAction() 'wav2lip' version 1: 2
I0804 00:10:30.280865 1 model_repository_manager.cc:1071] Unload() 'wav2lip' version 1
I0804 00:10:30.280872 1 model_repository_manager.cc:534] LiveBackendStates()
I0804 00:10:30.280882 1 server.cc:249] Timeout 30: Found 0 live models and 0 in-flight non-inference requests
I0804 00:10:30.280888 1 triton_backend_manager.cc:101] unloading backend 'pytorch'
I0804 00:10:30.280900 1 triton_backend_manager.cc:101] unloading backend 'tensorflow'
I0804 00:10:30.280911 1 triton_backend_manager.cc:101] unloading backend 'onnxruntime'
I0804 00:10:30.280949 1 triton_backend_manager.cc:101] unloading backend 'openvino'
error: creating server: Internal - failed to load all models
$ conda create -y -n test3 python=3.8.10
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: doneenvironment location: /home/user/miniconda3/envs/test3
added / updated specs:
The following packages will be downloaded:
package | build
---------------------------|-----------------
pip-21.2.2 | py38h06a4308_0 1.8 MB
------------------------------------------------------------
Total: 1.8 MBThe following NEW packages will be INSTALLED:
_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main _openmp_mutex pkgs/main/linux-64::_openmp_mutex-4.5-1_gnu ca-certificates pkgs/main/linux-64::ca-certificates-2021.7.5-h06a4308_1 certifi pkgs/main/linux-64::certifi-2021.5.30-py38h06a4308_0 ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.35.1-h7274673_9 libffi pkgs/main/linux-64::libffi-3.3-he6710b0_2 libgcc-ng pkgs/main/linux-64::libgcc-ng-9.3.0-h5101ec6_17 libgomp pkgs/main/linux-64::libgomp-9.3.0-h5101ec6_17 libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.3.0-hd4cf53a_17 ncurses pkgs/main/linux-64::ncurses-6.2-he6710b0_1 openssl pkgs/main/linux-64::openssl-1.1.1k-h27cfd23_0 pip pkgs/main/linux-64::pip-21.2.2-py38h06a4308_0 python pkgs/main/linux-64::python-3.8.10-h12debd9_8 readline pkgs/main/linux-64::readline-8.1-h27cfd23_0 setuptools pkgs/main/linux-64::setuptools-52.0.0-py38h06a4308_0 sqlite pkgs/main/linux-64::sqlite-3.36.0-hc218d9a_0 tk pkgs/main/linux-64::tk-8.6.10-hbc83047_0 wheel pkgs/main/noarch::wheel-0.36.2-pyhd3eb1b0_0 xz pkgs/main/linux-64::xz-5.2.5-h7b6447c_0 zlib pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3
Downloading and Extracting Packages pip-21.2.2 | 1.8 MB | ################################################################################################################################################################################################################################################################################ | 100% Preparing transaction: done Verifying transaction: done Executing transaction: done #
#
#
#
$ conda activate test3
$ pip install conda-pack librosa Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com Collecting conda-pack Downloading conda-pack-0.6.0.tar.gz (43 kB) |████████████████████████████████| 43 kB 1.8 MB/s Collecting librosa Downloading librosa-0.8.1-py3-none-any.whl (203 kB) |████████████████████████████████| 203 kB 3.1 MB/s Requirement already satisfied: setuptools in /home/user/workspace/miniconda3/envs/test3/lib/python3.8/site-packages (from conda-pack) (52.0.0.post20210125) Requirement already satisfied: decorator>=3.0.0 in /home/user/.local/lib/python3.8/site-packages (from librosa) (5.0.7) Collecting soundfile>=0.10.2 Downloading SoundFile-0.10.3.post1-py2.py3-none-any.whl (21 kB) Collecting audioread>=2.0.0 Downloading audioread-2.1.9.tar.gz (377 kB) |████████████████████████████████| 377 kB 13.7 MB/s Collecting packaging>=20.0 Downloading packaging-21.0-py3-none-any.whl (40 kB) |████████████████████████████████| 40 kB 23.8 MB/s Collecting resampy>=0.2.2 Downloading resampy-0.2.2.tar.gz (323 kB) |████████████████████████████████| 323 kB 13.4 MB/s Collecting numpy>=1.15.0 Downloading numpy-1.21.1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.8 MB) |████████████████████████████████| 15.8 MB 16.4 MB/s Collecting numba>=0.43.0 Downloading numba-0.53.1-cp38-cp38-manylinux2014_x86_64.whl (3.4 MB) |████████████████████████████████| 3.4 MB 86.8 MB/s Collecting scikit-learn!=0.19.0,>=0.14.0 Downloading scikit_learn-0.24.2-cp38-cp38-manylinux2010_x86_64.whl (24.9 MB) |████████████████████████████████| 24.9 MB 92.3 MB/s Collecting pooch>=1.0 Downloading pooch-1.4.0-py3-none-any.whl (51 kB) |████████████████████████████████| 51 kB 28.5 MB/s Collecting scipy>=1.0.0 Downloading scipy-1.7.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (28.4 MB) |████████████████████████████████| 28.4 MB 92.7 MB/s Collecting joblib>=0.14 Downloading joblib-1.0.1-py3-none-any.whl (303 kB) |████████████████████████████████| 303 kB 95.0 MB/s Collecting llvmlite<0.37,>=0.36.0rc1 Downloading llvmlite-0.36.0-cp38-cp38-manylinux2010_x86_64.whl (25.3 MB) |████████████████████████████████| 25.3 MB 89.6 MB/s Collecting pyparsing>=2.0.2 Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) |████████████████████████████████| 67 kB 79.2 MB/s Collecting appdirs Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB) Collecting requests Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB) |████████████████████████████████| 62 kB 41.0 MB/s Requirement already satisfied: six>=1.3 in /home/user/.local/lib/python3.8/site-packages (from resampy>=0.2.2->librosa) (1.16.0) Collecting threadpoolctl>=2.0.0 Downloading threadpoolctl-2.2.0-py3-none-any.whl (12 kB) Collecting cffi>=1.0 Downloading cffi-1.14.6-cp38-cp38-manylinux1_x86_64.whl (411 kB) |████████████████████████████████| 411 kB 91.4 MB/s Collecting pycparser Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB) |████████████████████████████████| 112 kB 94.9 MB/s Collecting idna<4,>=2.5 Downloading idna-3.2-py3-none-any.whl (59 kB) |████████████████████████████████| 59 kB 83.5 MB/s Collecting charset-normalizer~=2.0.0 Downloading charset_normalizer-2.0.4-py3-none-any.whl (36 kB) Collecting urllib3<1.27,>=1.21.1 Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB) |████████████████████████████████| 138 kB 94.1 MB/s Requirement already satisfied: certifi>=2017.4.17 in /home/user/workspace/miniconda3/envs/test3/lib/python3.8/site-packages (from requests->pooch>=1.0->librosa) (2021.5.30) Building wheels for collected packages: conda-pack, audioread, resampy Building wheel for conda-pack (setup.py) ... done Created wheel for conda-pack: filename=conda_pack-0.6.0-py2.py3-none-any.whl size=30893 sha256=02c3d67dec2dce2d8feacb7cc934b9b609e0ec0bc94e14f30eb6b3ec2b9d908f Stored in directory: /tmp/pip-ephem-wheel-cache-erk279xs/wheels/56/1b/9e/0da27a4c18349d8f048a8fe87d763d75d3098384e9fa285e45 Building wheel for audioread (setup.py) ... done Created wheel for audioread: filename=audioread-2.1.9-py3-none-any.whl size=23141 sha256=12caf2bd60d2636d5e0823e4485397ea217e4133c60c7ba3367c6fa742d21bff Stored in directory: /tmp/pip-ephem-wheel-cache-erk279xs/wheels/49/5a/e4/df590783499a992a88de6c0898991d1167453a3196d0d1eeb7 Building wheel for resampy (setup.py) ... done Created wheel for resampy: filename=resampy-0.2.2-py3-none-any.whl size=320718 sha256=97c6945550ea9c2cd2681d6d366d578d8f93ddae48a14864d645b1972de9f39f Stored in directory: /tmp/pip-ephem-wheel-cache-erk279xs/wheels/6f/d1/5d/f13da53b1dcbc2624ff548456c9ffb526c914f53c12c318bb4 Successfully built conda-pack audioread resampy Installing collected packages: urllib3, pyparsing, pycparser, numpy, llvmlite, idna, charset-normalizer, threadpoolctl, scipy, requests, packaging, numba, joblib, cffi, appdirs, soundfile, scikit-learn, resampy, pooch, audioread, librosa, conda-pack Successfully installed appdirs-1.4.4 audioread-2.1.9 cffi-1.14.6 charset-normalizer-2.0.4 conda-pack-0.6.0 idna-3.2 joblib-1.0.1 librosa-0.8.1 llvmlite-0.36.0 numba-0.53.1 numpy-1.21.1 packaging-21.0 pooch-1.4.0 pycparser-2.20 pyparsing-2.4.7 requests-2.26.0 resampy-0.2.2 scikit-learn-0.24.2 scipy-1.7.1 soundfile-0.10.3.post1 threadpoolctl-2.2.0 urllib3-1.26.6
$ python Python 3.8.10 (default, Jun 4 2021, 15:09:15) [GCC 7.5.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.
import librosa librosa.version '0.8.1' import decorator decorator.version '5.0.7'
$ conda-pack Collecting packages... Packing environment at '/home/user/miniconda3/envs/test3' to 'test3.tar.gz' [########################################] | 100% Completed | 17.2s
2. Output
```console
$ docker run --gpus='"device=3"' --shm-size=5G -p8000:8000 -p8001:8001 -p8002:8002 --rm -v /path/to/model_repository/wav2lip:/models --name tritonserver nvcr.io/nvidia/tritonserver:21.07-py3 tritonserver --model-repository=/models --log-verbose 20
=============================
== Triton Inference Server ==
=============================
NVIDIA Release 21.07 (build 24810355)
Copyright (c) 2018-2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
Various files include modifications (c) NVIDIA CORPORATION. All rights reserved.
This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
NOTE: Legacy NVIDIA Driver detected. Compatibility mode ENABLED.
I0804 00:20:37.731803 1 metrics.cc:290] Collecting metrics for GPU 0: Tesla V100-PCIE-16GB
I0804 00:20:37.732176 1 shared_library.cc:108] OpenLibraryHandle: /opt/tritonserver/backends/pytorch/libtriton_pytorch.so
I0804 00:20:38.033983 1 libtorch.cc:998] TRITONBACKEND_Initialize: pytorch
I0804 00:20:38.034022 1 libtorch.cc:1008] Triton TRITONBACKEND API version: 1.4
I0804 00:20:38.034028 1 libtorch.cc:1014] 'pytorch' TRITONBACKEND API version: 1.4
I0804 00:20:38.034099 1 shared_library.cc:108] OpenLibraryHandle: /opt/tritonserver/backends/tensorflow1/libtriton_tensorflow1.so
2021-08-04 00:20:38.187735: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
I0804 00:20:38.232102 1 tensorflow.cc:2169] TRITONBACKEND_Initialize: tensorflow
I0804 00:20:38.232139 1 tensorflow.cc:2179] Triton TRITONBACKEND API version: 1.4
I0804 00:20:38.232145 1 tensorflow.cc:2185] 'tensorflow' TRITONBACKEND API version: 1.4
I0804 00:20:38.232150 1 tensorflow.cc:2209] backend configuration:
{}
I0804 00:20:38.232208 1 shared_library.cc:108] OpenLibraryHandle: /opt/tritonserver/backends/onnxruntime/libtriton_onnxruntime.so
I0804 00:20:38.233521 1 onnxruntime.cc:1970] TRITONBACKEND_Initialize: onnxruntime
I0804 00:20:38.233544 1 onnxruntime.cc:1980] Triton TRITONBACKEND API version: 1.4
I0804 00:20:38.233550 1 onnxruntime.cc:1986] 'onnxruntime' TRITONBACKEND API version: 1.4
I0804 00:20:38.243055 1 shared_library.cc:108] OpenLibraryHandle: /opt/tritonserver/backends/openvino/libtriton_openvino.so
I0804 00:20:38.255174 1 openvino.cc:1193] TRITONBACKEND_Initialize: openvino
I0804 00:20:38.255216 1 openvino.cc:1203] Triton TRITONBACKEND API version: 1.4
I0804 00:20:38.255222 1 openvino.cc:1209] 'openvino' TRITONBACKEND API version: 1.4
I0804 00:20:38.673151 1 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7fe580000000' with size 268435456
I0804 00:20:38.673532 1 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864
I0804 00:20:38.674132 1 backend_factory.h:45] Create TritonBackendFactory
I0804 00:20:38.674156 1 plan_backend_factory.cc:49] Create PlanBackendFactory
I0804 00:20:38.674165 1 plan_backend_factory.cc:56] Registering TensorRT Plugins
I0804 00:20:38.674228 1 logging.cc:52] Registered plugin creator - ::GridAnchor_TRT version 1
I0804 00:20:38.674249 1 logging.cc:52] Registered plugin creator - ::GridAnchorRect_TRT version 1
I0804 00:20:38.674264 1 logging.cc:52] Registered plugin creator - ::NMS_TRT version 1
I0804 00:20:38.674274 1 logging.cc:52] Registered plugin creator - ::Reorg_TRT version 1
I0804 00:20:38.674284 1 logging.cc:52] Registered plugin creator - ::Region_TRT version 1
I0804 00:20:38.674300 1 logging.cc:52] Registered plugin creator - ::Clip_TRT version 1
I0804 00:20:38.674313 1 logging.cc:52] Registered plugin creator - ::LReLU_TRT version 1
I0804 00:20:38.674322 1 logging.cc:52] Registered plugin creator - ::PriorBox_TRT version 1
I0804 00:20:38.674330 1 logging.cc:52] Registered plugin creator - ::Normalize_TRT version 1
I0804 00:20:38.674338 1 logging.cc:52] Registered plugin creator - ::ScatterND version 1
I0804 00:20:38.674346 1 logging.cc:52] Registered plugin creator - ::RPROI_TRT version 1
I0804 00:20:38.674357 1 logging.cc:52] Registered plugin creator - ::BatchedNMS_TRT version 1
I0804 00:20:38.674366 1 logging.cc:52] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
I0804 00:20:38.674375 1 logging.cc:52] Registered plugin creator - ::FlattenConcat_TRT version 1
I0804 00:20:38.674385 1 logging.cc:52] Registered plugin creator - ::CropAndResize version 1
I0804 00:20:38.674395 1 logging.cc:52] Registered plugin creator - ::DetectionLayer_TRT version 1
I0804 00:20:38.674403 1 logging.cc:52] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
I0804 00:20:38.674411 1 logging.cc:52] Registered plugin creator - ::EfficientNMS_TRT version 1
I0804 00:20:38.674425 1 logging.cc:52] Registered plugin creator - ::Proposal version 1
I0804 00:20:38.674432 1 logging.cc:52] Registered plugin creator - ::ProposalLayer_TRT version 1
I0804 00:20:38.674441 1 logging.cc:52] Registered plugin creator - ::PyramidROIAlign_TRT version 1
I0804 00:20:38.674449 1 logging.cc:52] Registered plugin creator - ::ResizeNearest_TRT version 1
I0804 00:20:38.674458 1 logging.cc:52] Registered plugin creator - ::Split version 1
I0804 00:20:38.674466 1 logging.cc:52] Registered plugin creator - ::SpecialSlice_TRT version 1
I0804 00:20:38.674481 1 logging.cc:52] Registered plugin creator - ::InstanceNormalization_TRT version 1
I0804 00:20:38.674492 1 ensemble_backend_factory.cc:47] Create EnsembleBackendFactory
I0804 00:20:38.675952 1 model_repository_manager.cc:749] AsyncLoad() 'wav2lip'
I0804 00:20:38.676015 1 model_repository_manager.cc:988] TriggerNextAction() 'wav2lip' version 1: 1
I0804 00:20:38.676028 1 model_repository_manager.cc:1026] Load() 'wav2lip' version 1
I0804 00:20:38.676033 1 model_repository_manager.cc:1045] loading: wav2lip:1
I0804 00:20:38.776318 1 model_repository_manager.cc:1105] CreateInferenceBackend() 'wav2lip' version 1
I0804 00:20:38.776449 1 shared_library.cc:108] OpenLibraryHandle: /opt/tritonserver/backends/python/libtriton_python.so
I0804 00:20:38.778517 1 python.cc:1306] 'python' TRITONBACKEND API version: 1.4
I0804 00:20:38.778533 1 python.cc:1328] backend configuration:
{}
I0804 00:20:38.778545 1 python.cc:1405] shm-default-byte-size=67108864,shm-growth-byte-size=67108864,stub-timeout-seconds=30
I0804 00:20:38.778905 1 python.cc:1453] TRITONBACKEND_ModelInitialize: wav2lip (version 1)
I0804 00:20:38.779902 1 model_config_utils.cc:1521] ModelConfig 64-bit fields:
I0804 00:20:38.779918 1 model_config_utils.cc:1523] ModelConfig::dynamic_batching::default_queue_policy::default_timeout_microseconds
I0804 00:20:38.779923 1 model_config_utils.cc:1523] ModelConfig::dynamic_batching::max_queue_delay_microseconds
I0804 00:20:38.779927 1 model_config_utils.cc:1523] ModelConfig::dynamic_batching::priority_queue_policy::value::default_timeout_microseconds
I0804 00:20:38.779932 1 model_config_utils.cc:1523] ModelConfig::ensemble_scheduling::step::model_version
I0804 00:20:38.779936 1 model_config_utils.cc:1523] ModelConfig::input::dims
I0804 00:20:38.779941 1 model_config_utils.cc:1523] ModelConfig::input::reshape::shape
I0804 00:20:38.779945 1 model_config_utils.cc:1523] ModelConfig::instance_group::secondary_devices::device_id
I0804 00:20:38.779951 1 model_config_utils.cc:1523] ModelConfig::model_warmup::inputs::value::dims
I0804 00:20:38.779955 1 model_config_utils.cc:1523] ModelConfig::optimization::cuda::graph_spec::graph_lower_bound::input::value::dim
I0804 00:20:38.779960 1 model_config_utils.cc:1523] ModelConfig::optimization::cuda::graph_spec::input::value::dim
I0804 00:20:38.779965 1 model_config_utils.cc:1523] ModelConfig::output::dims
I0804 00:20:38.779970 1 model_config_utils.cc:1523] ModelConfig::output::reshape::shape
I0804 00:20:38.779975 1 model_config_utils.cc:1523] ModelConfig::sequence_batching::direct::max_queue_delay_microseconds
I0804 00:20:38.779981 1 model_config_utils.cc:1523] ModelConfig::sequence_batching::max_sequence_idle_microseconds
I0804 00:20:38.779985 1 model_config_utils.cc:1523] ModelConfig::sequence_batching::oldest::max_queue_delay_microseconds
I0804 00:20:38.779989 1 model_config_utils.cc:1523] ModelConfig::version_policy::specific::versions
I0804 00:20:38.780090 1 python.cc:1275] Using Python execution env /models/wav2lip/test3.tar.gz
I0804 00:20:38.781313 1 python.cc:1497] TRITONBACKEND_ModelInstanceInitialize: wav2lip_0 (GPU device 0)
I0804 00:20:38.782551 1 backend_model_instance.cc:105] Creating instance wav2lip_0 on GPU 0 (7.0) using artifact ''
I0804 00:20:46.091840 56 python.cc:918] Starting Python backend stub: export LD_LIBRARY_PATH=/tmp/python_env_zQcmpg/0/lib:$LD_LIBRARY_PATH; source /tmp/python_env_zQcmpg/0/bin/activate && exec /opt/tritonserver/backends/python/triton_python_backend_stub /models/wav2lip/1/model.py /wav2lip_0_GPU_0 67108864 67108864 1 /opt/tritonserver/backends/python
0804 00:20:46.782380 56 pb_stub.cc:639] ModuleNotFoundError: No module named 'decorator'
At:
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/_cache.py(40): wrapper
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/util/utils.py(230): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/util/__init__.py(83): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap>(961): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/core/notation.py(8): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap>(1050): _handle_fromlist
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/core/convert.py(7): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/core/__init__.py(5): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap>(1050): _handle_fromlist
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/__init__.py(211): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/models/wav2lip/1/model.py(13): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
I0804 00:20:46.782537 1 python.cc:1557] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0804 00:20:46.782761 1 python.cc:1476] TRITONBACKEND_ModelFinalize: delete model state
I0804 00:20:46.782792 1 triton_backend_manager.cc:101] unloading backend 'python'
I0804 00:20:46.782797 1 python.cc:1433] TRITONBACKEND_Finalize: Start
I0804 00:20:47.136235 1 python.cc:1438] TRITONBACKEND_Finalize: End
E0804 00:20:47.138378 1 model_repository_manager.cc:1215] failed to load 'wav2lip' version 1: Internal: ModuleNotFoundError: No module named 'decorator'
At:
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/_cache.py(40): wrapper
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/util/utils.py(230): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/util/__init__.py(83): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap>(961): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/core/notation.py(8): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap>(1050): _handle_fromlist
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/core/convert.py(7): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/core/__init__.py(5): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap>(1050): _handle_fromlist
/tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/__init__.py(211): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
/models/wav2lip/1/model.py(13): <module>
<frozen importlib._bootstrap>(219): _call_with_frames_removed
<frozen importlib._bootstrap_external>(848): exec_module
<frozen importlib._bootstrap>(686): _load_unlocked
<frozen importlib._bootstrap>(975): _find_and_load_unlocked
<frozen importlib._bootstrap>(991): _find_and_load
I0804 00:20:47.138412 1 model_repository_manager.cc:988] TriggerNextAction() 'wav2lip' version 1: 0
I0804 00:20:47.138421 1 model_repository_manager.cc:1003] no next action, trigger OnComplete()
I0804 00:20:47.138526 1 model_repository_manager.cc:594] VersionStates() 'wav2lip'
I0804 00:20:47.138602 1 model_repository_manager.cc:594] VersionStates() 'wav2lip'
I0804 00:20:47.138654 1 server.cc:504]
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+
I0804 00:20:47.138735 1 server.cc:543]
+-------------+-----------------------------------------------------------------+--------+
| Backend | Path | Config |
+-------------+-----------------------------------------------------------------+--------+
| tensorrt | <built-in> | {} |
| pytorch | /opt/tritonserver/backends/pytorch/libtriton_pytorch.so | {} |
| tensorflow | /opt/tritonserver/backends/tensorflow1/libtriton_tensorflow1.so | {} |
| onnxruntime | /opt/tritonserver/backends/onnxruntime/libtriton_onnxruntime.so | {} |
| openvino | /opt/tritonserver/backends/openvino/libtriton_openvino.so | {} |
+-------------+-----------------------------------------------------------------+--------+
I0804 00:20:47.138751 1 model_repository_manager.cc:570] BackendStates()
I0804 00:20:47.138848 1 server.cc:586]
+---------+---------+-----------------------------------------------------------------------------------------------+
| Model | Version | Status |
+---------+---------+-----------------------------------------------------------------------------------------------+
| wav2lip | 1 | UNAVAILABLE: Internal: ModuleNotFoundError: No module named 'decorator' |
| | | |
| | | At: |
| | | /tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/_cache.py(40): wrapper |
| | | /tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/util/utils.py(230): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(848): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | /tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/util/__init__.py(83): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(848): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap>(961): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | /tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/core/notation.py(8): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(848): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap>(1050): _handle_fromlist |
| | | /tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/core/convert.py(7): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(848): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | /tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/core/__init__.py(5): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(848): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap>(1050): _handle_fromlist |
| | | /tmp/python_env_zQcmpg/0/lib/python3.8/site-packages/librosa/__init__.py(211): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(848): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
| | | /models/wav2lip/1/model.py(13): <module> |
| | | <frozen importlib._bootstrap>(219): _call_with_frames_removed |
| | | <frozen importlib._bootstrap_external>(848): exec_module |
| | | <frozen importlib._bootstrap>(686): _load_unlocked |
| | | <frozen importlib._bootstrap>(975): _find_and_load_unlocked |
| | | <frozen importlib._bootstrap>(991): _find_and_load |
+---------+---------+-----------------------------------------------------------------------------------------------+
I0804 00:20:47.138954 1 tritonserver.cc:1718]
+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Option | Value |
+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| server_id | triton |
| server_version | 2.12.0 |
| server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data statistics |
| model_repository_path[0] | /models |
| model_control_mode | MODE_NONE |
| strict_model_config | 1 |
| pinned_memory_pool_byte_size | 268435456 |
| cuda_memory_pool_byte_size{0} | 67108864 |
| min_supported_compute_capability | 6.0 |
| strict_readiness | 1 |
| exit_timeout | 30 |
+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I0804 00:20:47.138966 1 server.cc:234] Waiting for in-flight requests to complete.
I0804 00:20:47.138970 1 model_repository_manager.cc:694] AsyncUnload() 'wav2lip'
I0804 00:20:47.138975 1 model_repository_manager.cc:988] TriggerNextAction() 'wav2lip' version 1: 2
I0804 00:20:47.138980 1 model_repository_manager.cc:1071] Unload() 'wav2lip' version 1
I0804 00:20:47.138988 1 model_repository_manager.cc:534] LiveBackendStates()
I0804 00:20:47.138994 1 server.cc:249] Timeout 30: Found 0 live models and 0 in-flight non-inference requests
I0804 00:20:47.139001 1 triton_backend_manager.cc:101] unloading backend 'pytorch'
I0804 00:20:47.139010 1 triton_backend_manager.cc:101] unloading backend 'tensorflow'
I0804 00:20:47.139023 1 triton_backend_manager.cc:101] unloading backend 'onnxruntime'
I0804 00:20:47.139062 1 triton_backend_manager.cc:101] unloading backend 'openvino'
error: creating server: Internal - failed to load all modelsI was able to reproduce this. Looks like some of the packages in your environment are installed outside the conda environment. You can check that with:
import librosa
print(librosa.__file__)I tried conda install -c conda-forge librosa and then used conda-pack and this issue was resolved.
ghost
commented
3 years ago conda does not seem to isolate the environment completely like virtualenv
The local path of a user is included in os.sys.path so conda does not install packages if they are already in there.
A workaround would be exporting the environment variable PYTHONNOUSERSITE before activating conda environment
$ export PYTHONNOUSERSITE=True
$ conda activate test2
$ python
Python 3.8.10 (default, Jun 4 2021, 15:09:15)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> from pprint import pprint
>>> pprint(os.sys.path)
['',
'/home/user/miniconda3/envs/test2/lib/python38.zip',
'/home/user/miniconda3/envs/test2/lib/python3.8',
'/home/user/miniconda3/envs/test2/lib/python3.8/lib-dynload',
'/home/user/miniconda3/envs/test2/lib/python3.8/site-packages']
$ conda deactivate
# Note that the local path of user is included
$ unset PYTHONNOUSERSITE
$ conda activate test2
$ python
Python 3.8.10 (default, Jun 4 2021, 15:09:15)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> from pprint import pprint
>>> pprint(os.sys.path)
['',
'/home/user/miniconda3/envs/test2/lib/python38.zip',
'/home/user/miniconda3/envs/test2/lib/python3.8',
'/home/user/miniconda3/envs/test2/lib/python3.8/lib-dynload',
'/home/user/.local/lib/python3.8/site-packages',
'/home/user/miniconda3/envs/test2/lib/python3.8/site-packages']I guess we should mention it in README?
Tabrizian
commented
3 years ago @ihcho9088 Thanks for pointing this out! I'll add a note in README about this.
Tabrizian
commented
3 years ago Looks like it is something related to the older versions of conda. Inside the Triton 21.07 container I was not able to reproduce the same os.sys.path:
root@767d49b41c50:/opt/tritonserver/qa/L0_backend_python# conda activate test2
(test2) root@767d49b41c50:/opt/tritonserver/qa/L0_backend_python# python3
Python 3.8.11 (default, Aug 3 2021, 15:09:35)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> print(os.sys.path)
['', '/opt/tritonserver/qa/L0_backend_python/miniconda/envs/test2/lib/python38.zip', '/opt/tritonserver/qa/L0_backend_python/miniconda/envs/test2/lib/python3.8', '/opt/tritonserver/qa/L0_backend_python/miniconda/envs/test2/lib/python3.8/lib-dynload', '/opt/tritonserver/qa/L0_backend_python/miniconda/envs/test2/lib/python3.8/site-packages']I have not set the PYTHONNOUSERSITE env variable.
ghost
commented
3 years ago @Tabrizian You wouldn't find the local path if you haven’t installed any packages yet in local.
Try installing packages, say torch with --user flag.
# In triton server as root
$ ls -al /root/.local
ls: cannot access '/root/.local': No such file or directory
$ pip install torch --user
…
Install packages
…
$ ls -al /root/.local/lib/python3.8/site-packages/
…
bunch of packages
…
# Now activate conda environment and verify local path is shown
@ihcho9088 Thanks for the reply! I added a note about this in the README.
gioipv
commented
3 years ago Hello, @Tabrizian @ihcho9088
I met this issues. but even when i set: export PYTHONNOUSERSITE=True before activate conda env to conda-pack .
And i run triton docker with command:
docker run --gpus=1 --shm-size=5G -p8111:8111 -p8222:8222 -p8333:8333 --rm \ -v /home/gioipv/workspaces/ekyc_glasses/triton/model_repo2:/models --name tritonserver nvcr.io/nvidia/tritonserver:20.11-py3 tritonserver --model-repository=/models --log-verbose 20
It raise an error:
`I0811 02:36:54.797002 1 model_config_utils.cc:1559]
ModelConfig::sequence_batching::oldest::max_queue_delay_microseconds
I0811 02:36:54.797014 1 model_config_utils.cc:1559] ModelConfig::version_policy::specific::versions
I0811 02:36:54.797350 1 python.cc:696] TRITONBACKEND_ModelInstanceInitialize: model1_0 (GPU device 0)
Traceback (most recent call last):
File "/opt/tritonserver/backends/python/startup.py", line 360, in
python_host = PythonHost(module_path=FLAGS.model_path)
File "/opt/tritonserver/backends/python/startup.py", line 161, in init
spec.loader.exec_module(module)
File "
File "
File "/models/model1/1/model.py", line 3, in
import librosa
ModuleNotFoundError: No module named 'librosa'
I0811 02:36:59.806248 1 python.cc:1017] TRITONBACKEND_ModelInstanceFinalize: delete instance state
I0811 02:36:59.806907 1 python.cc:418] GRPC shutdown complete
I0811 02:36:59.806976 1 python.cc:672] TRITONBACKEND_ModelFinalize: delete model state
I0811 02:36:59.807015 1 triton_backend_manager.cc:130] unloading backend 'python'
I0811 02:36:59.807024 1 python.cc:627] TRITONBACKEND_Finalize: Start`
And i checked my conda env with python command:
Python 3.8.11 (default, Aug 3 2021, 15:09:35) [GCC 7.5.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information. import librosa print(librosa.__file__) /home/gioipv/workspaces/miniconda3/envs/test2/lib/python3.8/site-packages/librosa/__init__.py
could you help me with this
Tabrizian
commented
3 years ago @gioipv Did you set that environment before installing the packages as well?
gioipv
commented
3 years ago @Tabrizian
Thank you for you help.
Yes, i seted before install the packages.
(export PYTHONNOUSERSITE=True).
But I think I see some my problem in https://github.com/triton-inference-server/server/issues/3222
Description
import librosainmodel.pyraisesModuleNotFoundError: No module name d 'decorator'Triton Information
nvcr.io/nvidia/tritonserver:21.07-py3
To Reproduce
$ conda create -y -n test2 python=3.8 Collecting package metadata (current_repodata.json): done Solving environment: done
Package Plan
environment location: /home/user/miniconda3/envs/test2
added / updated specs:
The following packages will be downloaded:
The following NEW packages will be INSTALLED:
_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main _openmp_mutex pkgs/main/linux-64::_openmp_mutex-4.5-1_gnu ca-certificates pkgs/main/linux-64::ca-certificates-2021.7.5-h06a4308_1 certifi pkgs/main/linux-64::certifi-2021.5.30-py38h06a4308_0 ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.35.1-h7274673_9 libffi pkgs/main/linux-64::libffi-3.3-he6710b0_2 libgcc-ng pkgs/main/linux-64::libgcc-ng-9.3.0-h5101ec6_17 libgomp pkgs/main/linux-64::libgomp-9.3.0-h5101ec6_17 libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.3.0-hd4cf53a_17 ncurses pkgs/main/linux-64::ncurses-6.2-he6710b0_1 openssl pkgs/main/linux-64::openssl-1.1.1k-h27cfd23_0 pip pkgs/main/linux-64::pip-21.1.3-py38h06a4308_0 python pkgs/main/linux-64::python-3.8.11-h12debd9_0_cpython readline pkgs/main/linux-64::readline-8.1-h27cfd23_0 setuptools pkgs/main/linux-64::setuptools-52.0.0-py38h06a4308_0 sqlite pkgs/main/linux-64::sqlite-3.36.0-hc218d9a_0 tk pkgs/main/linux-64::tk-8.6.10-hbc83047_0 wheel pkgs/main/noarch::wheel-0.36.2-pyhd3eb1b0_0 xz pkgs/main/linux-64::xz-5.2.5-h7b6447c_0 zlib pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3
Downloading and Extracting Packages python-3.8.11 | 18.2 MB | ################################ | 100% Preparing transaction: done Verifying transaction: done Executing transaction: done #
To activate this environment, use
#
$ conda activate test2
#
To deactivate an active environment, use
#
$ conda deactivate
$ conda activate test2
$ pip install conda-pack librosa Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com Collecting conda-pack Downloading conda-pack-0.6.0.tar.gz (43 kB) |ââââââââââââââââââââââââââââââââ| 43 kB 1.8 MB/s Collecting librosa Downloading librosa-0.8.1-py3-none-any.whl (203 kB) |ââââââââââââââââââââââââââââââââ| 203 kB 3.1 MB/s Requirement already satisfied: setuptools in /home/andy/workspace/miniconda3/envs/test2/lib/python3.8/site-packages (from conda-pack) (52.0.0.post20210125) Collecting scikit-learn!=0.19.0,>=0.14.0 Downloading scikit_learn-0.24.2-cp38-cp38-manylinux2010_x86_64.whl (24.9 MB) |ââââââââââââââââââââââââââââââââ| 24.9 MB 22.4 MB/s Collecting joblib>=0.14 Downloading joblib-1.0.1-py3-none-any.whl (303 kB) |ââââââââââââââââââââââââââââââââ| 303 kB 80.6 MB/s Collecting numba>=0.43.0 Downloading numba-0.53.1-cp38-cp38-manylinux2014_x86_64.whl (3.4 MB) |ââââââââââââââââââââââââââââââââ| 3.4 MB 77.5 MB/s Collecting resampy>=0.2.2 Downloading resampy-0.2.2.tar.gz (323 kB) |ââââââââââââââââââââââââââââââââ| 323 kB 78.1 MB/s Collecting packaging>=20.0 Downloading packaging-21.0-py3-none-any.whl (40 kB) |ââââââââââââââââââââââââââââââââ| 40 kB 54.0 MB/s Collecting scipy>=1.0.0 Downloading scipy-1.7.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (28.4 MB) |ââââââââââââââââââââââââââââââââ| 28.4 MB 95.6 MB/s Collecting numpy>=1.15.0 Downloading numpy-1.21.1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.8 MB) |ââââââââââââââââââââââââââââââââ| 15.8 MB 93.8 MB/s Collecting pooch>=1.0 Downloading pooch-1.4.0-py3-none-any.whl (51 kB) |ââââââââââââââââââââââââââââââââ| 51 kB 24.1 MB/s Requirement already satisfied: decorator>=3.0.0 in /home/andy/.local/lib/python3.8/site-packages (from librosa) (5.0.7) Collecting soundfile>=0.10.2 Downloading SoundFile-0.10.3.post1-py2.py3-none-any.whl (21 kB) Collecting audioread>=2.0.0 Downloading audioread-2.1.9.tar.gz (377 kB) |ââââââââââââââââââââââââââââââââ| 377 kB 86.6 MB/s Collecting llvmlite<0.37,>=0.36.0rc1 Downloading llvmlite-0.36.0-cp38-cp38-manylinux2010_x86_64.whl (25.3 MB) |ââââââââââââââââââââââââââââââââ| 25.3 MB 89.6 MB/s Collecting pyparsing>=2.0.2 Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) |ââââââââââââââââââââââââââââââââ| 67 kB 60.7 MB/s Collecting appdirs Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB) Collecting requests Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB) |ââââââââââââââââââââââââââââââââ| 62 kB 35.8 MB/s Requirement already satisfied: six>=1.3 in /home/andy/.local/lib/python3.8/site-packages (from resampy>=0.2.2->librosa) (1.16.0) Collecting threadpoolctl>=2.0.0 Downloading threadpoolctl-2.2.0-py3-none-any.whl (12 kB) Collecting cffi>=1.0 Downloading cffi-1.14.6-cp38-cp38-manylinux1_x86_64.whl (411 kB) |ââââââââââââââââââââââââââââââââ| 411 kB 89.6 MB/s Collecting pycparser Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB) |ââââââââââââââââââââââââââââââââ| 112 kB 89.8 MB/s Collecting urllib3<1.27,>=1.21.1 Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB) |ââââââââââââââââââââââââââââââââ| 138 kB 89.7 MB/s Requirement already satisfied: certifi>=2017.4.17 in /home/andy/workspace/miniconda3/envs/test2/lib/python3.8/site-packages (from requests->pooch>=1.0->librosa) (2021.5.30) Collecting idna<4,>=2.5 Downloading idna-3.2-py3-none-any.whl (59 kB) |ââââââââââââââââââââââââââââââââ| 59 kB 75.7 MB/s Collecting charset-normalizer~=2.0.0 Downloading charset_normalizer-2.0.4-py3-none-any.whl (36 kB) Building wheels for collected packages: conda-pack, audioread, resampy Building wheel for conda-pack (setup.py) ... done Created wheel for conda-pack: filename=conda_pack-0.6.0-py2.py3-none-any.whl size=30893 sha256=3a483159f1e919ff3c1104b411958c3a3b66c8c0e9037bc72ebff9d180712a22 Stored in directory: /tmp/pip-ephem-wheel-cache-1o200pri/wheels/56/1b/9e/0da27a4c18349d8f048a8fe87d763d75d3098384e9fa285e45 Building wheel for audioread (setup.py) ... done Created wheel for audioread: filename=audioread-2.1.9-py3-none-any.whl size=23141 sha256=c3eefc897b7d45a17d4c07404e4a26a269b45efb8b7b162f78bb16e543c08877 Stored in directory: /tmp/pip-ephem-wheel-cache-1o200pri/wheels/49/5a/e4/df590783499a992a88de6c0898991d1167453a3196d0d1eeb7 Building wheel for resampy (setup.py) ... done Created wheel for resampy: filename=resampy-0.2.2-py3-none-any.whl size=320718 sha256=bbfb6df8f7e542e9637f85fbe0452da1ca2ae08a004d4b6f7b8839bbaa2e8f29 Stored in directory: /tmp/pip-ephem-wheel-cache-1o200pri/wheels/6f/d1/5d/f13da53b1dcbc2624ff548456c9ffb526c914f53c12c318bb4 Successfully built conda-pack audioread resampy Installing collected packages: urllib3, pyparsing, pycparser, numpy, llvmlite, idna, charset-normalizer, threadpoolctl, scipy, requests, packaging, numba, joblib, cffi, appdirs, soundfile, scikit-learn, resampy, pooch, audioread, librosa, conda-pack Successfully installed appdirs-1.4.4 audioread-2.1.9 cffi-1.14.6 charset-normalizer-2.0.4 conda-pack-0.6.0 idna-3.2 joblib-1.0.1 librosa-0.8.1 llvmlite-0.36.0 numba-0.53.1 numpy-1.21.1 packaging-21.0 pooch-1.4.0 pycparser-2.20 pyparsing-2.4.7 requests-2.26.0 resampy-0.2.2 scikit-learn-0.24.2 scipy-1.7.1 soundfile-0.10.3.post1 threadpoolctl-2.2.0 urllib3-1.26.6
$ python Python 3.8.11 (default, Aug 3 2021, 15:09:35) [GCC 7.5.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.
$ conda-pack Collecting packages... Packing environment at '/home/user/miniconda3/envs/test2' to 'test2.tar.gz' [########################################] | 100% Completed | 15.7s
input [ { name: "INPUT0" data_type: TYPE_STRING dims: [ -1 ] }, { name: "INPUT1" data_type: TYPE_STRING dims: [ -1 ] } ]
output [ { name: "OUTPUT0" data_type: TYPE_STRING dims: [ -1 ] } ]
max_batch_size: 1
instance_group [{ count: 1, kind: KIND_GPU }]
version_policy: { latest { num_versions : 1 }}
parameters: { key: "EXECUTION_ENV_PATH", value: {string_value: "/models/wav2lip/test2.tar.gz"} }