triple-Mu / YOLOv8-TensorRT

YOLOv8 using TensorRT accelerate !
MIT License
1.23k stars 210 forks source link

How it Modify for YOLOV8 pose for multi classes #190

Open faithmaker opened 6 months ago

faithmaker commented 6 months ago

The model learned with multi-class(3) is performed normally with the yolov8 cli Predict command. I confirmed through debugging that only one class objects was detected when using the example here.

*. you recommed. You should modify pose postprocess. Default is only one class so that the score is only for person. But if enable multiclass you should use argmax and max to find the highest score and label. => Where should I fix it?

*. I check it.

YOLOv8-TensorRT\csrc\pose\normal\include\yolov8-pose.hpp .... void YOLOv8_pose::postprocess(std::vector& objs, float score_thres, float iou_thres, int topk) { ... auto kps_ptr = row_ptr + 5;

// classnum = 3 auto maxSPtr = std::max_element(kps_ptr, kps_ptr + 3); // auto maxSPtr = std::max_element(kps_ptr+2, kps_ptr + 3); // auto maxSPtr = std::max_element(kps_ptr+3, kps_ptr + 3);

float score = *scores_ptr;
if(score > score_thres) 
    {
    label = maxSPtr - scores_ptr;

// Not Detect label cout << func << ";label=" << label << endl; .....

int cnt = 0; /*

ifdef BATCHED_NMS

cv::dnn::NMSBoxesBatched(bboxes, scores, labels, score_thresd, iou_thres, indices);

else

cv::dnn::NMSBoxes(bboxes, scores, score_thres, iou_thres, indices);

endif

for(auto& i : indices) */ // 8 object Detect, but almost same position and size. for(int i = 0; i < bboxes.size(); i++) { if(cnt >= topk) break; Object obj; obj.rect = bboxes[i]; obj.prob = scores[i]; obj.label = labels[i]; obj.kps = kpss[i]; objs.push_back(obj); cnt += 1; } } ....

triple-Mu commented 6 months ago

The model learned with multi-class(3) is performed normally with the yolov8 cli Predict command. I confirmed through debugging that only one class objects was detected when using the example here.

*. you recommed. You should modify pose postprocess. Default is only one class so that the score is only for person. But if enable multiclass you should use argmax and max to find the highest score and label. => Where should I fix it?

*. I check it.

YOLOv8-TensorRT\csrc\pose\normal\include\yolov8-pose.hpp .... void YOLOv8_pose::postprocess(std::vector& objs, float score_thres, float iou_thres, int topk) { ... auto kps_ptr = row_ptr + 5;

// classnum = 3 auto maxSPtr = std::max_element(kps_ptr, kps_ptr + 3); // auto maxSPtr = std::max_element(kps_ptr+2, kps_ptr + 3); // auto maxSPtr = std::max_element(kps_ptr+3, kps_ptr + 3);

float score = *scores_ptr;
if(score > score_thres) 
  {
  label = maxSPtr - scores_ptr;

// Not Detect label cout << func << ";label=" << label << endl; .....

int cnt = 0; /* #ifdef BATCHED_NMS cv::dnn::NMSBoxesBatched(bboxes, scores, labels, score_thresd, iou_thres, indices); #else cv::dnn::NMSBoxes(bboxes, scores, score_thres, iou_thres, indices); #endif

for(auto& i : indices) */ // 8 object Detect, but almost same position and size. for(int i = 0; i < bboxes.size(); i++) { if(cnt >= topk) break; Object obj; obj.rect = bboxes[i]; obj.prob = scores[i]; obj.label = labels[i]; obj.kps = kpss[i]; objs.push_back(obj); cnt += 1; } } ....

Sure you should find the top label by scores_ptr. And make the Object.label to this label. And do postprocess and draw label on image.

faithmaker commented 6 months ago

Even if all objects were detected and output, the location could only be estimated for one class. The 17 location points are also wrong. Still, objects of other classes are not even detected. Objects are not detected properly before labeling.

*. YoloV8-TensorRt Program result(8 object Detect, but almost same position and size) GetDetectBoxQue;Over1;Class=0;Center=1124,1218;Box=839,880;cv::Size=569,675 TrackYoloV8TrtPose;0;kps_s=198.936 TrackYoloV8TrtPose;1;kps_s=203.058 TrackYoloV8TrtPose;2;kps_s=195.693 TrackYoloV8TrtPose;3;kps_s=208.787 TrackYoloV8TrtPose;4;kps_s=189.63 TrackYoloV8TrtPose;5;kps_s=216.226 TrackYoloV8TrtPose;6;kps_s=177.447 TrackYoloV8TrtPose;7;kps_s=227.426 TrackYoloV8TrtPose;8;kps_s=157.746 TrackYoloV8TrtPose;9;kps_s=211.389 TrackYoloV8TrtPose;10;kps_s=174.692 TrackYoloV8TrtPose;11;kps_s=203.97 TrackYoloV8TrtPose;12;kps_s=173.19 TrackYoloV8TrtPose;13;kps_s=197.799 TrackYoloV8TrtPose;14;kps_s=165.32 TrackYoloV8TrtPose;15;kps_s=190.509 TrackYoloV8TrtPose;16;kps_s=156.924 GetDetectBoxQue;Over1;Class=0;Center=1124,1218;Box=840,879;cv::Size=569,679 TrackYoloV8TrtPose;0;kps_s=198.781 TrackYoloV8TrtPose;1;kps_s=203.272 TrackYoloV8TrtPose;2;kps_s=195.489 TrackYoloV8TrtPose;3;kps_s=209.107 TrackYoloV8TrtPose;4;kps_s=189.734 TrackYoloV8TrtPose;5;kps_s=215.908 TrackYoloV8TrtPose;6;kps_s=177.684 TrackYoloV8TrtPose;7;kps_s=229.06 TrackYoloV8TrtPose;8;kps_s=157.92 TrackYoloV8TrtPose;9;kps_s=211.557 TrackYoloV8TrtPose;10;kps_s=174.08 TrackYoloV8TrtPose;11;kps_s=203.287 TrackYoloV8TrtPose;12;kps_s=172.927 TrackYoloV8TrtPose;13;kps_s=197.695 TrackYoloV8TrtPose;14;kps_s=165.009 TrackYoloV8TrtPose;15;kps_s=189.77 TrackYoloV8TrtPose;16;kps_s=157.612 GetDetectBoxQue;Over1;Class=0;Center=1123,1218;Box=838,878;cv::Size=570,679 TrackYoloV8TrtPose;0;kps_s=198.684 TrackYoloV8TrtPose;1;kps_s=203.246 TrackYoloV8TrtPose;2;kps_s=195.461 TrackYoloV8TrtPose;3;kps_s=209.105 TrackYoloV8TrtPose;4;kps_s=189.693 TrackYoloV8TrtPose;5;kps_s=215.875 TrackYoloV8TrtPose;6;kps_s=177.634 TrackYoloV8TrtPose;7;kps_s=228.773 TrackYoloV8TrtPose;8;kps_s=157.929 TrackYoloV8TrtPose;9;kps_s=210.562 TrackYoloV8TrtPose;10;kps_s=173.788 TrackYoloV8TrtPose;11;kps_s=203.522 TrackYoloV8TrtPose;12;kps_s=173.2 TrackYoloV8TrtPose;13;kps_s=197.458 TrackYoloV8TrtPose;14;kps_s=165.086 TrackYoloV8TrtPose;15;kps_s=189.363 TrackYoloV8TrtPose;16;kps_s=157.905 GetDetectBoxQue;Over1;Class=0;Center=1122,1218;Box=836,879;cv::Size=571,678 TrackYoloV8TrtPose;0;kps_s=198.432 TrackYoloV8TrtPose;1;kps_s=202.424 TrackYoloV8TrtPose;2;kps_s=195.297 TrackYoloV8TrtPose;3;kps_s=208.145 TrackYoloV8TrtPose;4;kps_s=189.213 TrackYoloV8TrtPose;5;kps_s=215.082 TrackYoloV8TrtPose;6;kps_s=177.583 TrackYoloV8TrtPose;7;kps_s=226.372 TrackYoloV8TrtPose;8;kps_s=157.573 TrackYoloV8TrtPose;9;kps_s=211.008 TrackYoloV8TrtPose;10;kps_s=174.175 TrackYoloV8TrtPose;11;kps_s=203.245 TrackYoloV8TrtPose;12;kps_s=173.486 TrackYoloV8TrtPose;13;kps_s=197.998 TrackYoloV8TrtPose;14;kps_s=165.479 TrackYoloV8TrtPose;15;kps_s=190.475 TrackYoloV8TrtPose;16;kps_s=157.531 GetDetectBoxQue;Over1;Class=0;Center=1124,1218;Box=838,879;cv::Size=571,679 TrackYoloV8TrtPose;0;kps_s=198.532 TrackYoloV8TrtPose;1;kps_s=202.854 TrackYoloV8TrtPose;2;kps_s=195.302 TrackYoloV8TrtPose;3;kps_s=208.825 TrackYoloV8TrtPose;4;kps_s=189.529 TrackYoloV8TrtPose;5;kps_s=215.159 TrackYoloV8TrtPose;6;kps_s=177.943 TrackYoloV8TrtPose;7;kps_s=228.456 TrackYoloV8TrtPose;8;kps_s=157.458 TrackYoloV8TrtPose;9;kps_s=210.602 TrackYoloV8TrtPose;10;kps_s=173.761 TrackYoloV8TrtPose;11;kps_s=202.909 TrackYoloV8TrtPose;12;kps_s=173.135 TrackYoloV8TrtPose;13;kps_s=197.75 TrackYoloV8TrtPose;14;kps_s=165.267 TrackYoloV8TrtPose;15;kps_s=189.916 TrackYoloV8TrtPose;16;kps_s=157.957 GetDetectBoxQue;Over1;Class=0;Center=1123,1218;Box=837,880;cv::Size=572,677 TrackYoloV8TrtPose;0;kps_s=198.699 TrackYoloV8TrtPose;1;kps_s=203.128 TrackYoloV8TrtPose;2;kps_s=195.515 TrackYoloV8TrtPose;3;kps_s=209.187 TrackYoloV8TrtPose;4;kps_s=189.707 TrackYoloV8TrtPose;5;kps_s=215.665 TrackYoloV8TrtPose;6;kps_s=178.23 TrackYoloV8TrtPose;7;kps_s=228.964 TrackYoloV8TrtPose;8;kps_s=157.863 TrackYoloV8TrtPose;9;kps_s=210.612 TrackYoloV8TrtPose;10;kps_s=173.778 TrackYoloV8TrtPose;11;kps_s=203.795 TrackYoloV8TrtPose;12;kps_s=173.664 TrackYoloV8TrtPose;13;kps_s=198.012 TrackYoloV8TrtPose;14;kps_s=164.791 TrackYoloV8TrtPose;15;kps_s=189.592 TrackYoloV8TrtPose;16;kps_s=157.448 GetDetectBoxQue;Over1;Class=0;Center=1122,1218;Box=838,877;cv::Size=568,681 TrackYoloV8TrtPose;0;kps_s=198.826 TrackYoloV8TrtPose;1;kps_s=203.418 TrackYoloV8TrtPose;2;kps_s=195.816 TrackYoloV8TrtPose;3;kps_s=208.668 TrackYoloV8TrtPose;4;kps_s=190.348 TrackYoloV8TrtPose;5;kps_s=215.541 TrackYoloV8TrtPose;6;kps_s=177.889 TrackYoloV8TrtPose;7;kps_s=228.712 TrackYoloV8TrtPose;8;kps_s=157.739 TrackYoloV8TrtPose;9;kps_s=210.797 TrackYoloV8TrtPose;10;kps_s=174.608 TrackYoloV8TrtPose;11;kps_s=204.512 TrackYoloV8TrtPose;12;kps_s=172.62 TrackYoloV8TrtPose;13;kps_s=197.175 TrackYoloV8TrtPose;14;kps_s=165.003 TrackYoloV8TrtPose;15;kps_s=188.71 TrackYoloV8TrtPose;16;kps_s=156.816 GetDetectBoxQue;Over1;Class=0;Center=1122,1217;Box=836,877;cv::Size=572,680 TrackYoloV8TrtPose;0;kps_s=198.887 TrackYoloV8TrtPose;1;kps_s=203.393 TrackYoloV8TrtPose;2;kps_s=195.857 TrackYoloV8TrtPose;3;kps_s=208.635 TrackYoloV8TrtPose;4;kps_s=190.238 TrackYoloV8TrtPose;5;kps_s=215.461 TrackYoloV8TrtPose;6;kps_s=178.049 TrackYoloV8TrtPose;7;kps_s=228.574 TrackYoloV8TrtPose;8;kps_s=158.14 TrackYoloV8TrtPose;9;kps_s=210.797 TrackYoloV8TrtPose;10;kps_s=174.844 TrackYoloV8TrtPose;11;kps_s=204.366 TrackYoloV8TrtPose;12;kps_s=173.327 TrackYoloV8TrtPose;13;kps_s=197.403 TrackYoloV8TrtPose;14;kps_s=165.003 TrackYoloV8TrtPose;15;kps_s=189.178 TrackYoloV8TrtPose;16;kps_s=157.538 GetDetectBoxQue;Over1;Class=0;Center=1122,1217;Box=838,877;cv::Size=569,680 TrackYoloV8TrtPose;0;kps_s=198.956 TrackYoloV8TrtPose;1;kps_s=203.516 TrackYoloV8TrtPose;2;kps_s=195.814 TrackYoloV8TrtPose;3;kps_s=208.714 TrackYoloV8TrtPose;4;kps_s=190.404 TrackYoloV8TrtPose;5;kps_s=215.592 TrackYoloV8TrtPose;6;kps_s=178.2 TrackYoloV8TrtPose;7;kps_s=228.806 TrackYoloV8TrtPose;8;kps_s=157.672 TrackYoloV8TrtPose;9;kps_s=211.009 TrackYoloV8TrtPose;10;kps_s=174.543 TrackYoloV8TrtPose;11;kps_s=204.407 TrackYoloV8TrtPose;12;kps_s=172.916 TrackYoloV8TrtPose;13;kps_s=197.595 TrackYoloV8TrtPose;14;kps_s=165.03 TrackYoloV8TrtPose;15;kps_s=189.387 TrackYoloV8TrtPose;16;kps_s=157.959 TensorRtResultError

*. Ultralytics Cli result UltralyticsResult640

triple-Mu commented 1 month ago

Do you have further questions?