fkunn1326 / openpose-editor

Openpose Editor for AUTOMATIC1111's stable-diffusion-webui
MIT License
1.73k stars 207 forks source link

improve detectImage #116

Closed nhjydywd closed 1 year ago

nhjydywd commented 1 year ago

The original detectImage function may result in confusing output when not all keypoints are detected, like the following picture:

original

Such confusing output was reported as a bug in issue #115

Here is how this problem was caused: in the original detectImage function, keypoints that were not detected will be assigned a random coordinate, as follows:

if (Number.isInteger(subset[i]) && subset[i] >= 0){
    li.push(candidate[subset[i]])
}else{
    const ra_width = Math.floor(Math.random() * openpose_editor_canvas.width)
    const ra_height = Math.floor(Math.random() * openpose_editor_canvas.height)
    li.push([ra_width, ra_height])
}

As a result, we see undetected keypoints scattered randomly on the image, and the detected pose is confusing.

I made some change to fix this. Specifically, the position of each undetected keypoint will be determined by its adjacent detected keypoints, rather than being randomly determined. This will improve the detection results.

// bfs propagate
while(li.some(([x,y])=>x===-1&&y===-1)){
    for(i=0;i<li.length;i++){
        if(li[i][0]===-1){
            continue;
        }
        for(j=0;j<li.length;j++){
            if(li[j][0]===-1 && kp_connect(i,j)){
                x = li[i][0] + default_relative_keypoints[i][j][0]
                y = li[i][1] + default_relative_keypoints[i][j][1]
                x = Math.min(Math.max(x, 0), openpose_editor_canvas.width);
                y = Math.min(Math.max(y, 0), openpose_editor_canvas.height);
                li[j] = [x,y];
            }
        }
    }
}

For more information, please see the code difference in the pull request.

Here is the detection result using the new code, it's much less confusing: improved

In a word, this pull request improved the detectImage function to avoid confusing output. Please consider accept it.

fkunn1326 commented 1 year ago

LGTM