search

isarsoft / yolov4-triton-tensorrt

This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server
http://www.isarsoft.com
Other
278 stars 64 forks source link

GRPC: unable to provide 'prob' in GPU, will use CPU #20

Closed chiyukunpeng closed 3 years ago

chiyukunpeng commented 3 years ago

Hi, I want to infer videos with triton20.08 using GPU instead of CPU. Can you help me?

server

I1222 02:48:37.420736 1 plan_backend.cc:1652] Running yolov5x_0_gpu1 with 1 requests
I1222 02:48:37.420799 1 plan_backend.cc:2384] Optimization profile default [0] is selected for yolov5x_0_gpu1
I1222 02:48:37.420847 1 pinned_memory_manager.cc:130] pinned memory allocation: size 4435968, addr 0x7f8f78000090
I1222 02:48:37.421611 1 plan_backend.cc:1911] Context with profile default [0] is being executed for yolov5x_0_gpu1
I1222 02:48:37.423902 1 infer_response.cc:139] add response output: output: prob, type: FP32, shape: [1,6001,1,1]
I1222 02:48:37.423941 1 grpc_server.cc:2151] GRPC: unable to provide 'prob' in GPU, will use CPU
I1222 02:48:37.423958 1 grpc_server.cc:2162] GRPC: using buffer for 'prob', size: 24004, addr: 0x7f8caa0658f0
I1222 02:48:37.423978 1 pinned_memory_manager.cc:130] pinned memory allocation: size 24004, addr 0x7f8f7843b0a0
I1222 02:48:37.424035 1 pinned_memory_manager.cc:157] pinned memory deallocation: addr 0x7f8f78000090
I1222 02:48:37.433026 1 pinned_memory_manager.cc:157] pinned memory deallocation: addr 0x7f8f7843b0a0
I1222 02:48:37.433072 1 grpc_server.cc:3158] ModelInferHandler::InferResponseComplete, 251 step ISSUED
I1222 02:48:37.433100 1 grpc_server.cc:2197] GRPC free: size 24004, addr 0x7f8caa0658f0
I1222 02:48:37.433262 1 grpc_server.cc:2736] ModelInferHandler::InferRequestComplete
I1222 02:48:37.433277 1 grpc_server.cc:3007] Process for ModelInferHandler, rpc_ok=1, 251 step COMPLETE
I1222 02:48:37.433298 1 grpc_server.cc:2071] Done for ModelInferHandler, 251

client

Frame 246: 62 raw boxes, 3 objects
car:     0.94
car:     0.93
car:     0.93
time:    31.0ms
Frame 247: 63 raw boxes, 3 objects
car:     0.94
car:     0.93
car:     0.93
time:    28.4ms
Frame 248: 73 raw boxes, 3 objects
car:     0.93
car:     0.92
car:     0.92
time:    27.8ms
philipp-schmidt commented 3 years ago

You are already using GPU, you can safele ignore this warning. Check usage of your GPU with nvidia-smi.