Closed chiyukunpeng closed 3 years ago
By how much is your precision decreased? How do you measure mAP in both cases?
@philipp-schmidt hi, I unframed two output videos. I found that the video after deployment has false detections but there are no false detections before deployment.
Hi, Yes, I have experienced similar problems and reported them in 2 issues in the triton repo. Can you share more details so I can add them to the issue?
E.g. are you using multiple clients at the same time? How fast are you retrieving results? How is the error looking? Only missing detections or wrong detections as well?
I believe the following issues are related to this problem:
https://github.com/triton-inference-server/server/issues/2231 https://github.com/triton-inference-server/server/issues/2135
And probably also this one: https://github.com/triton-inference-server/server/issues/2322
I would ask you to give me as much info that you can give away so I can get a proper bug report going. How many GPUs do you use?
Please also try without FP16 optimization:
@philipp-schmidt Hi, it is my pleasure to share more details.
Are you using multiple clients at the same time? No.
How fast are you retrieving results?
It infers videos with CPU
. I want to infer it with GPU
. Can you help me? Thanks!
Frame 248: 73 raw boxes, 3 objects
car: 0.93
car: 0.92
car: 0.92
time: 27.8ms
How is the error looking? Only missing detections or wrong detections as well? My errors include missing detections and wrong detections.
How many GPUs do you use? one 2080 GPU.
Please also try without FP16 optimization Missing detections and wrong detections decreases, but they still exist.
How do you measure your mAP and can you share an example of an image with correct detections before deployment and wrong detections after? How do you validate without triton?
before deployment after deployment
before deployment after deployment
I don't see what's wrong here. Truck and Chemical are swapped because your class indices have to be changed in the python client code, search for the COCOLabels in the code.
The other image might simply be due to different detection thresholds. Make sure they are identical and compare again.
Thanks!
hi, I infer surveillance videos with
yolov5x
according to https://github.com/wang-xinyu/tensorrtx/tree/master/yolov5.mAP@.50
of that is94.8%
.After deploying
yolov5x
withtriton20.08
, I found thatmAP@.50
decreased.Do you know the reasons for that? Thanks!
triton client output
postprocess func