the difference between openpose output values of keypoints and test_data.pkl

[Bo396543018]

Thank you for your great work! when I process my data，i find the range of openpose output is inconsistent with the values provided in test_data.pkl, how can i get the same range?

[bharat-b7]

You need to normalize the openpose output (divide by image size).

[Bo396543018]

You need to normalize the openpose output (divide by image size). Thanks.Besides divide image size, the values in test_data.pkl range from [-1, 1], does this mean that it needs to be further normalized to [-1, 1]?

[neonb88]

Thanks.Besides divide image size, the values in test_data.pkl range from [-1, 1], does this mean that it needs to be further normalized to [-1, 1]?

@Bo396543018 In the end, did you normalize to [-1, 1] or [0, 1]?

[Bo396543018]

@neonb88 yes，i used code from this repo octopus

def openpose_from_file(file, resolution=(1080, 1080), person=0):
    with open(file) as f:
        data = json.load(f)['people'][person]

        pose = np.array(data['pose_keypoints_2d']).reshape(-1, 3)
        pose[:, 2] /= np.expand_dims(np.mean(pose[:, 2][pose[:, 2] > 0.1]), -1)
        pose = pose * np.array([2. / resolution[1], -2. / resolution[0], 1.]) + np.array([-1., 1., 0.])
        pose[:, 0] *= 1. * resolution[1] / resolution[0]

        face = np.array(data['face_keypoints_2d']).reshape(-1, 3)
        face = face * np.array([2. / resolution[1], -2. / resolution[0], 1.]) + np.array([-1., 1., 0.])
        face[:, 0] *= 1. * resolution[1] / resolution[0]

        return pose, face

[andrewjong]

Seems like it's normalized to [0, 1] then? FYI you can set the keypoint_scale flag to 3 in OpenPose to automatically scale your keypoint output.

[neonb88]

Seems like it's normalized to [0, 1] then?

@andrewjong Maybe I'm doing something wrong, but I'm getting [-1,1] for x and y values.

[neonb88]

Also, @bharat-b7 , why are the confidence values from your supplied test_data.pkl file repeatedly above 1? (ie. out of the natural range of openpose confidence levels, [0,1])

Is this how the neural network was originally trained? Did Alldieck et al. train octopus the same way? I acknowledge that perhaps there is no difference in the outcome, but being 153% sure of something definitely isn't intuitive, and I've never seen OpenPose output such a high confidence value

Please confirm; maybe it's my mistake. :smile:

import pickle as pkl

if __name__=="__main__":
  test_data = pkl.load( open('test_data.pkl', 'rb')   ,   encoding='latin1')
  CONFIDENCE=2
  NUM_IMGS=8
  conf_max = -float('inf')
  conf_min = float('inf')

  for i in range(NUM_IMGS):
    # max 
    curr_conf=test_data['J_2d_{}'.format(i)][:,:,CONFIDENCE].max()
    if curr_conf > conf_max:
      conf_max = curr_conf
# at the end of running this code, `conf_max` is 1.536375805901848

If you used python to prepare the contents of test_data.pkl (or if those python files are somewhere in octopus), please let us know. Perhaps it was an issue with the line of code pose[:, 2] /= np.expand_dims(np.mean(pose[:, 2][pose[:, 2] > 0.1]), -1)?

[dev2021-ctrl]

@Bo396543018 can u please tell me how you read test_data.pkl and know the values present there.