liyunfei0411
commented
1 year ago +-------------+---------+-----------------------------------------------------------------------------------------------------------------------------------+ | Model | Version | Status | +-------------+---------+-----------------------------------------------------------------------------------------------------------------------------------+ | postprocess | 1 | READY | | preprocess | 1 | READY | | runtime | 1 | UNAVAILABLE: Not found: unable to load backend library: libopencv_video.so.3.4: cannot open shared object file: No such file or d | | | | irectory | +-------------+---------+-----------------------------------------------------------------------------------------------------------------------------------+
NiZerin
root@ubuntu:/serving/examples/vision/detection/paddledetection/serving# fastdeployserver --model-repository=./models/ I0906 03:13:30.949881 244 metrics.cc:298] Collecting metrics for GPU 0: NVIDIA GeForce RTX 3070 I0906 03:13:31.064709 244 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7f4f02000000' with size 268435456 I0906 03:13:31.064851 244 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864 I0906 03:13:31.066167 244 model_repository_manager.cc:1022] loading: postprocess:1 I0906 03:13:31.166556 244 model_repository_manager.cc:1022] loading: preprocess:1 I0906 03:13:31.172910 244 python.cc:1875] TRITONBACKEND_ModelInstanceInitialize: postprocess_0 (CPU device 0) I0906 03:13:31.266781 244 model_repository_manager.cc:1022] loading: runtime:1 model_config: {'name': 'postprocess', 'platform': '', 'backend': 'python', 'version_policy': {'latest': {'num_versions': 1}}, 'max_batch_size': 0, 'input': [{'name': 'post_input1', 'data_type': 'TYPE_FP32', 'format': 'FORMAT_NONE', 'dims': [-1, 6], 'is_shape_tensor': False, 'allow_ragged_batch': False}, {'name': 'post_input2', 'data_type': 'TYPE_INT32', 'format': 'FORMAT_NONE', 'dims': [-1], 'is_shape_tensor': False, 'allow_ragged_batch': False}], 'output': [{'name': 'post_output', 'data_type': 'TYPE_STRING', 'dims': [-1], 'label_filename': '', 'is_shape_tensor': False}], 'batch_input': [], 'batch_output': [], 'optimization': {'priority': 'PRIORITY_DEFAULT', 'input_pinned_memory': {'enable': True}, 'output_pinned_memory': {'enable': True}, 'gather_kernel_buffer_threshold': 0, 'eager_batching': False}, 'instance_group': [{'name': 'postprocess_0', 'kind': 'KIND_CPU', 'count': 1, 'gpus': [], 'secondary_devices': [], 'profile': [], 'passive': False, 'host_policy': ''}], 'default_model_filename': '', 'cc_model_filenames': {}, 'metric_tags': {}, 'parameters': {}, 'model_warmup': []} postprocess input names: ['post_input1', 'post_input2'] postprocess output names: ['post_output'] I0906 03:13:32.080263 244 model_repository_manager.cc:1183] successfully loaded 'postprocess' version 1 I0906 03:13:32.080383 244 python.cc:1875] TRITONBACKEND_ModelInstanceInitialize: preprocess_0 (CPU device 0) model_config: {'name': 'preprocess', 'platform': '', 'backend': 'python', 'version_policy': {'latest': {'num_versions': 1}}, 'max_batch_size': 0, 'input': [{'name': 'preprocess_input', 'data_type': 'TYPE_UINT8', 'format': 'FORMAT_NONE', 'dims': [-1, -1, -1, 3], 'is_shape_tensor': False, 'allow_ragged_batch': False}], 'output': [{'name': 'preprocess_output1', 'data_type': 'TYPE_FP32', 'dims': [-1, 3, -1, -1], 'label_filename': '', 'is_shape_tensor': False}, {'name': 'preprocess_output2', 'data_type': 'TYPE_FP32', 'dims': [-1, 2], 'label_filename': '', 'is_shape_tensor': False}, {'name': 'preprocess_output3', 'data_type': 'TYPE_FP32', 'dims': [-1, 2], 'label_filename': '', 'is_shape_tensor': False}], 'batch_input': [], 'batch_output': [], 'optimization': {'priority': 'PRIORITY_DEFAULT', 'input_pinned_memory': {'enable': True}, 'output_pinned_memory': {'enable': True}, 'gather_kernel_buffer_threshold': 0, 'eager_batching': False}, 'instance_group': [{'name': 'preprocess_0', 'kind': 'KIND_CPU', 'count': 1, 'gpus': [], 'secondary_devices': [], 'profile': [], 'passive': False, 'host_policy': ''}], 'default_model_filename': '', 'cc_model_filenames': {}, 'metric_tags': {}, 'parameters': {}, 'model_warmup': []} preprocess input names: ['preprocess_input'] preprocess output names: ['preprocess_output1', 'preprocess_output2', 'preprocess_output3'] [INFO] fastdeploy/vision/common/processors/transform.cc(45)::FuseNormalizeCast Normalize and Cast are fused to Normalize in preprocessing pipeline. [INFO] fastdeploy/vision/common/processors/transform.cc(93)::FuseNormalizeHWC2CHW Normalize and HWC2CHW are fused to NormalizeAndPermute in preprocessing pipeline. [INFO] fastdeploy/vision/common/processors/transform.cc(159)::FuseNormalizeColorConvert BGR2RGB and NormalizeAndPermute are fused to NormalizeAndPermute with swap_rb=1 I0906 03:13:33.008112 244 model_repository_manager.cc:1183] successfully loaded 'preprocess' version 1 E0906 03:13:33.008500 244 model_repository_manager.cc:1186] failed to load 'runtime' version 1: Not found: unable to load backend library: libopencv_video.so.3.4: cannot open shared object file: No such file or directory E0906 03:13:33.008573 244 model_repository_manager.cc:1375] Invalid argument: ensemble 'ppdet' depends on 'runtime' which has no loaded version I0906 03:13:33.008606 244 server.cc:522] +------------------+------+ | Repository Agent | Path | +------------------+------+ +------------------+------+
I0906 03:13:33.008621 244 server.cc:549] +---------+-------------------------------------------------------+--------+ | Backend | Path | Config | +---------+-------------------------------------------------------+--------+ | python | /opt/tritonserver/backends/python/libtriton_python.so | {} | +---------+-------------------------------------------------------+--------+
I0906 03:13:33.008653 244 server.cc:592] +-------------+---------+-----------------------------------------------------------------------------------------------------------------------------------+ | Model | Version | Status | +-------------+---------+-----------------------------------------------------------------------------------------------------------------------------------+ | postprocess | 1 | READY | | preprocess | 1 | READY | | runtime | 1 | UNAVAILABLE: Not found: unable to load backend library: libopencv_video.so.3.4: cannot open shared object file: No such file or d | | | | irectory | +-------------+---------+-----------------------------------------------------------------------------------------------------------------------------------+
I0906 03:13:33.008719 244 tritonserver.cc:1920] +----------------------------------+--------------------------------------------------------------------------------------------------------------------------+ | Option | Value | +----------------------------------+--------------------------------------------------------------------------------------------------------------------------+ | server_id | triton | | server_version | 2.15.0 | | server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy modelconfiguration system | | | shared_memory cuda_shared_memory binary_tensor_data statistics | | model_repository_path[0] | ./models/ | | model_control_mode | MODE_NONE | | strict_model_config | 1 | | rate_limit | OFF | | pinned_memory_pool_byte_size | 268435456 | | cuda_memory_pool_byte_size{0} | 67108864 | | response_cache_byte_size | 0 | | min_supported_compute_capability | 6.0 | | strict_readiness | 1 | | exit_timeout | 30 | +----------------------------------+--------------------------------------------------------------------------------------------------------------------------+
I0906 03:13:33.008740 244 server.cc:252] Waiting for in-flight requests to complete. I0906 03:13:33.008746 244 model_repository_manager.cc:1055] unloading: preprocess:1 I0906 03:13:33.008765 244 model_repository_manager.cc:1055] unloading: postprocess:1 I0906 03:13:33.008782 244 server.cc:267] Timeout 30: Found 2 live models and 0 in-flight non-inference requests I0906 03:13:34.008874 244 server.cc:267] Timeout 29: Found 2 live models and 0 in-flight non-inference requests Cleaning up... Cleaning up... I0906 03:13:34.132314 244 model_repository_manager.cc:1166] successfully unloaded 'preprocess' version 1 I0906 03:13:34.132739 244 model_repository_manager.cc:1166] successfully unloaded 'postprocess' version 1 I0906 03:13:35.009028 244 server.cc:267] Timeout 28: Found 0 live models and 0 in-flight non-inference requests error: creating server: Internal - failed to load all models