Closed chiyukunpeng closed 3 years ago
Hi, I want to infer videos with triton20.08 using GPU instead of CPU. Can you help me?
triton20.08
GPU
CPU
server
I1222 02:48:37.420736 1 plan_backend.cc:1652] Running yolov5x_0_gpu1 with 1 requests I1222 02:48:37.420799 1 plan_backend.cc:2384] Optimization profile default [0] is selected for yolov5x_0_gpu1 I1222 02:48:37.420847 1 pinned_memory_manager.cc:130] pinned memory allocation: size 4435968, addr 0x7f8f78000090 I1222 02:48:37.421611 1 plan_backend.cc:1911] Context with profile default [0] is being executed for yolov5x_0_gpu1 I1222 02:48:37.423902 1 infer_response.cc:139] add response output: output: prob, type: FP32, shape: [1,6001,1,1] I1222 02:48:37.423941 1 grpc_server.cc:2151] GRPC: unable to provide 'prob' in GPU, will use CPU I1222 02:48:37.423958 1 grpc_server.cc:2162] GRPC: using buffer for 'prob', size: 24004, addr: 0x7f8caa0658f0 I1222 02:48:37.423978 1 pinned_memory_manager.cc:130] pinned memory allocation: size 24004, addr 0x7f8f7843b0a0 I1222 02:48:37.424035 1 pinned_memory_manager.cc:157] pinned memory deallocation: addr 0x7f8f78000090 I1222 02:48:37.433026 1 pinned_memory_manager.cc:157] pinned memory deallocation: addr 0x7f8f7843b0a0 I1222 02:48:37.433072 1 grpc_server.cc:3158] ModelInferHandler::InferResponseComplete, 251 step ISSUED I1222 02:48:37.433100 1 grpc_server.cc:2197] GRPC free: size 24004, addr 0x7f8caa0658f0 I1222 02:48:37.433262 1 grpc_server.cc:2736] ModelInferHandler::InferRequestComplete I1222 02:48:37.433277 1 grpc_server.cc:3007] Process for ModelInferHandler, rpc_ok=1, 251 step COMPLETE I1222 02:48:37.433298 1 grpc_server.cc:2071] Done for ModelInferHandler, 251
client
Frame 246: 62 raw boxes, 3 objects car: 0.94 car: 0.93 car: 0.93 time: 31.0ms Frame 247: 63 raw boxes, 3 objects car: 0.94 car: 0.93 car: 0.93 time: 28.4ms Frame 248: 73 raw boxes, 3 objects car: 0.93 car: 0.92 car: 0.92 time: 27.8ms
You are already using GPU, you can safele ignore this warning. Check usage of your GPU with nvidia-smi.
Hi, I want to infer videos with
triton20.08
usingGPU
instead ofCPU
. Can you help me?server
client