search

DirtyHarryLYL / DJ-RN

As a part of HAKE project (HAKE-3D). Code for our CVPR2020 paper "Detailed 2D-3D Joint Representation for Human-Object Interaction".
Apache License 2.0
100 stars 13 forks source link

while the pkl file exists if not os.path.exists(os.path.join(args.res, 'results/HICO_train2015_%08d/%03d.pkl' % (key, item[6]))): continue doesn't work as expected and generate_3D_obj_GT.py creates no obj file #59

Closed monacv closed 3 years ago

monacv commented 3 years ago

Could you please explain what you are trying to do here?

For me, it never get past the continue hence no obj files gets saved by generate_3D_obj_GT.py file


        if not os.path.exists(os.path.join(args.res, 'results/HICO_train2015_%08d/%03d.pkl' % (key, item[6]))):
            continue

So, in my result folder I have something like: /scratch3/research/data/smplify-x/results/HICO_train2015_00035938/002.pkl as well as Screenshot from 2021-06-02 23-43-29

why for this specific image for example it doesn't exits? while the pkl file clearly are there? this is a recurrence to previously created issue here by me https://github.com/DirtyHarryLYL/DJ-RN/issues/57

Thanks a lot for your time

monacv commented 3 years ago

for this specific image I am not sure why openpose is outputting -1 and hence item[6] is -1 while smplify-x is outputting 3 pkl files as well as 3 obj files. I am not sure as well why person_id in below json is -1? "person_id": [-1]

For example for image: HICO_train2015_00035938.jpg

HICO_train2015_00035938_keypoints.json I have this as for openpose json: {"version": 1.3, "people": [{"person_id": [-1], "pose_keypoints_2d": [57.9641, 111.445, 0.881001, 142.763, 116.634, 0.566139, 120.575, 123.188, 0.335315, 0, 0, 0, 0, 0, 0, 163.635, 112.745, 0.577791, 257.619, 153.243, 0.658064, 255.059, 200.169, 0.648252, 277.216, 166.262, 0.394841, 260.229, 174.091, 0.24376, 316.329, 258.93, 0.55903, 371.193, 287.636, 0.772677, 296.745, 161.014, 0.454072, 307.23, 287.668, 0.780739, 371.12, 360.756, 0.792142, 0, 0, 0, 65.7808, 99.7428, 0.87955, 0, 0, 0, 99.7096, 90.5571, 0.855261, 332.041, 389.434, 0.620924, 346.369, 390.711, 0.589634, 381.674, 365.946, 0.750708, 371.208, 320.293, 0.505647, 372.495, 318.945, 0.466091, 381.66, 286.324, 0.641935], "face_keypoints_2d": [58.6666, 99.5718, 0.337398, 56.9838, 107.313, 0.275102, 56.9838, 111.688, 0.304404, 58.6666, 116.4, 0.312173, 59.3397, 121.448, 0.222463, 64.7247, 123.467, 0.193109, 65.0612, 128.179, 0.0892597, 72.4655, 128.516, 0.322151, 75.158, 131.881, 0.360997, 76.5042, 133.901, 0.401987, 86.6009, 131.208, 0.392083, 94.6783, 126.16, 0.259222, 96.6977, 120.438, 0.128183, 98.717, 114.044, 0.0980449, 99.7267, 102.601, 0.23225, 96.6977, 98.8987, 0.324048, 92.659, 92.5041, 0.254468, 64.7247, 96.2063, 0.322828, 62.7053, 92.8407, 0.322985, 62.0322, 91.831, 0.32167, 60.0128, 93.1772, 0.337786, 59.3397, 94.86, 0.431715, 59.3397, 96.2063, 0.54064, 60.686, 95.1966, 0.517185, 62.7053, 93.8504, 0.52864, 64.3881, 93.5138, 0.579677, 66.0709, 93.8504, 0.415149, 59.3397, 101.255, 0.663571, 59.3397, 106.64, 0.843622, 57.6569, 110.342, 0.799286, 56.6473, 115.054, 0.905376, 61.0225, 114.717, 0.556646, 61.0225, 117.073, 0.728161, 62.0322, 118.419, 0.954386, 63.0419, 118.083, 1.00775, 64.7247, 118.083, 0.881342, 62.3687, 98.8987, 0.495479, 63.3784, 98.8987, 0.649781, 63.715, 98.8987, 0.629877, 62.3687, 99.5718, 0.638739, 62.7053, 99.5718, 0.594943, 62.7053, 99.2353, 0.523889, 64.7247, 101.591, 0.804124, 65.0612, 100.918, 0.732871, 67.0806, 99.9084, 0.683304, 69.4365, 99.5718, 0.551932, 67.7537, 101.591, 0.728493, 66.4074, 101.928, 0.699773, 66.0709, 117.746, 0.4432, 64.7247, 119.429, 0.65708, 64.3881, 120.775, 0.899198, 64.7247, 122.458, 0.861238, 65.7343, 122.458, 0.825656, 69.773, 123.467, 0.887069, 74.1483, 124.141, 0.818842, 71.7924, 126.496, 0.801617, 69.773, 127.843, 0.737964, 67.7537, 126.833, 0.734378, 67.4171, 125.487, 0.762849, 66.744, 121.785, 0.633261, 66.4074, 118.756, 0.573568, 65.0612, 122.458, 0.788842, 65.3978, 123.467, 0.815631, 67.4171, 123.804, 0.881167, 72.802, 124.141, 0.867508, 67.7537, 124.141, 0.841426, 66.744, 123.804, 0.773265, 65.3978, 122.794, 0.828289, 63.715, 98.8987, 0.636518, 67.0806, 100.918, 0.705949], "hand_left_keypoints_2d": [254.59, 166.475, 0.0128916, 259.095, 174.734, 0.0109877, 275.238, 195.007, 0.0436856, 256.467, 202.515, 0.0540605, 246.706, 200.638, 0.0244403, 276.74, 190.502, 0.0977094, 243.327, 207.771, 0.319159, 229.812, 207.02, 0.387962, 220.051, 205.894, 0.38971, 261.723, 202.14, 0.0772808, 248.207, 209.648, 0.083518, 227.184, 206.27, 0.0477953, 220.051, 204.392, 0.0985676, 264.726, 201.764, 0.065809, 262.098, 204.768, 0.063959, 254.214, 194.256, 0.0383744, 221.552, 205.894, 0.0658882, 276.74, 196.133, 0.093049, 266.603, 202.891, 0.0564522, 229.061, 207.396, 0.0532289, 221.928, 207.396, 0.0667439], "hand_right_keypoints_2d": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "pose_keypoints_3d": [], "face_keypoints_3d": [], "hand_left_keypoints_3d": [], "hand_right_keypoints_3d": []}, {"person_id": [-1], "pose_keypoints_2d": [405.083, 33.0975, 0.774228, 390.797, 46.1115, 0.870972, 358.131, 38.3963, 0.79124, 334.642, 81.4243, 0.730241, 358.076, 89.3253, 0.780133, 426.004, 47.5225, 0.79948, 420.752, 108.787, 0.704497, 399.838, 150.635, 0.693596, 378.974, 121.873, 0.362139, 358.136, 120.597, 0.355047, 0, 0, 0, 0, 0, 0, 397.29, 123.206, 0.336227, 0, 0, 0, 0, 0, 0, 401.117, 25.2634, 0.835216, 411.65, 25.2802, 0.894839, 386.852, 20.0861, 0.760195, 422.009, 23.9713, 0.574254, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "face_keypoints_2d": [386.699, 42.6334, 0.0357428, 386.699, 42.6334, 0.00145885, 379.675, 31.9057, 0.0481681, 379.675, 39.0575, 0.0159125, 385.039, 44.8045, 0.0123209, 386.571, 45.1877, 0.00977995, 399.342, 32.1611, 0.0305844, 399.342, 32.1611, 0.0010076, 405.345, 45.8262, 0.00700358, 379.675, 42.6334, 0.000356225, 379.675, 42.6334, 0.0240272, 396.533, 26.7972, 0.0157209, 394.362, 47.4865, 0.0660838, 396.66, 47.2311, 0.0288701, 396.405, 46.8479, 0.00906317, 403.429, 39.4407, 0.0316275, 402.024, 46.7202, 0.0181353, 405.472, 27.5635, 0.0300557, 404.834, 27.6912, 0.0371969, 404.962, 27.6912, 0.00124072, 388.104, 47.7419, 0.000786942, 394.106, 44.6768, 0.000190774, 391.935, 46.8479, 6.92366e-05, 394.872, 47.8696, 6.2646e-05, 394.745, 47.8696, -2.97324e-05, 394.872, 47.8696, -0.000148643, 402.024, 46.7202, 2.66088e-05, 395.383, 44.6768, 0.002107, 395.383, 44.6768, 0.00138464, 395.383, 44.6768, 0.000219852, 392.063, 46.8479, 0.000116311, 380.824, 50.9347, 0.00104144, 380.824, 50.807, 0.0006818, 380.952, 50.807, 0.000132659, 393.851, 47.7419, 9.50485e-05, 402.918, 38.0358, 6.50355e-05, 405.983, 27.4358, 0.0214228, 394.362, 44.4214, 0.0112777, 394.106, 44.6768, 0.0101227, 392.957, 45.3154, 0.0186807, 392.957, 44.6768, 0.0256384, 394.106, 44.6768, 0.00317194, 395.256, 44.6768, -1.12273e-06, 395.128, 44.6768, -8.94273e-05, 394.872, 44.6768, -5.46898e-05, 394.617, 44.5491, 1.03678e-05, 394.234, 44.6768, -2.6261e-06, 395.128, 44.6768, -6.08431e-05, 392.19, 46.7202, 0.00016417, 406.494, 27.5635, 0.000277783, 406.622, 26.0309, 0.000320222, 405.856, 27.4358, 0.00138135, 393.212, 47.6142, 0.000172502, 394.745, 47.7419, 0.00159408, 396.022, 40.4624, 0.0149847, 394.745, 47.8696, 0.00120321, 404.834, 27.4358, 7.04378e-05, 388.104, 47.7419, 8.60682e-05, 393.34, 47.7419, 0.000246439, 399.215, 32.2888, 8.78373e-05, 386.443, 45.3154, 6.41017e-05, 405.089, 27.4358, 0.00553336, 405.089, 27.4358, 0.0105332, 393.34, 47.6142, 0.000815648, 397.043, 40.5901, 0.0113345, 393.212, 47.6142, 0.000495597, 392.318, 46.9756, 0.0135946, 392.318, 47.1034, 0.00238538, 393.851, 44.6768, 0.0305176, 394.234, 44.6768, -2.29019e-05], "hand_left_keypoints_2d": [398.124, 172.115, 0.0375786, 399.929, 179.107, 0.0219314, 386.395, 182.717, 0.0344767, 378.951, 181.814, 0.0217975, 376.244, 182.942, 0.00994337, 391.809, 172.791, 0.0233263, 383.011, 182.942, 0.0322387, 380.079, 184.07, 0.0608929, 373.988, 184.296, 0.0431416, 390.906, 174.37, 0.0153632, 385.267, 182.04, 0.0342, 380.981, 183.619, 0.0725217, 373.988, 184.521, 0.0484571, 392.485, 175.047, 0.0152405, 386.62, 184.07, 0.0291985, 380.755, 182.265, 0.0580187, 373.988, 184.747, 0.053978, 386.846, 184.521, 0.0159185, 386.395, 184.747, 0.0181199, 382.109, 185.198, 0.0339674, 373.763, 184.747, 0.0256079], "hand_right_keypoints_2d": [362.212, 91.7528, 0.074261, 363.651, 86.7174, 0.124294, 372.643, 81.682, 0.183349, 384.871, 84.5593, 0.429535, 390.087, 87.6165, 0.501926, 387.929, 94.0906, 0.0521512, 388.288, 94.0906, 0.0970264, 382.893, 89.4149, 0.104974, 380.196, 87.6165, 0.0676021, 384.871, 97.3277, 0.0620378, 386.49, 95.7091, 0.0733144, 382.354, 92.6519, 0.050302, 379.836, 91.3931, 0.0449114, 383.792, 98.9462, 0.0555765, 383.792, 98.5865, 0.0781226, 381.095, 95.3495, 0.0504854, 380.555, 92.8318, 0.0291262, 377.858, 99.4857, 0.0705516, 384.692, 99.8454, 0.0673594, 380.915, 99.8454, 0.0575847, 379.476, 98.4067, 0.0279868], "pose_keypoints_3d": [], "face_keypoints_3d": [], "hand_left_keypoints_3d": [], "hand_right_keypoints_3d": []}, {"person_id": [-1], "pose_keypoints_2d": [290.246, 3.73224, 0.725745, 273.286, 18.7665, 0.809743, 244.601, 17.4849, 0.820343, 223.665, 77.5544, 0.866059, 265.495, 87.9876, 0.854265, 301.975, 21.3628, 0.784412, 298.147, 78.8347, 0.76792, 262.822, 98.3382, 0.384836, 260.236, 101.009, 0.371049, 239.358, 99.6718, 0.38228, 0, 0, 0, 0, 0, 0, 281.109, 104.917, 0.345319, 0, 0, 0, 0, 0, 0, 283.744, 3.74593, 0.369786, 296.754, 3.74246, 0.449136, 269.359, 3.74316, 0.261996, 304.573, 3.74395, 0.24057, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "face_keypoints_2d": [303.773, 10.0184, 0.0468899, 266.293, 15.8639, 0.0145566, 267.21, 17.6978, 0.00644388, 295.521, 14.3739, 0.0156497, 277.411, 18.9586, 0.00331748, 277.411, 18.9586, 0.00552712, 299.647, 12.6546, 0.0149746, 300.106, 11.7377, 0.000836328, 265.032, 10.5915, 0.000761458, 264.918, 10.5915, 0.00033194, 264.918, 10.5915, 0.0240148, 264.918, 10.5915, 0.00167532, 277.411, 19.0733, 0.0152756, 278.786, 18.9586, 0.0107116, 266.637, 15.6347, 0.00499685, 266.637, 15.6347, 0.00694939, 305.722, -11.4153, 0.00872587, 304.805, 7.84065, 0.00288554, 303.773, 10.0184, 0.000130604, 303.773, 10.0184, 0.000132667, 300.22, 11.6231, 0.00115908, 303.773, 10.0184, 3.70073e-05, 264.918, 10.5915, -6.89919e-05, 305.722, -11.4153, -0.00011321, 305.607, -11.3007, -5.25713e-05, 300.106, 11.6231, -0.000169668, 305.722, -11.4153, 6.06714e-06, 305.722, 28.3574, 0.0016495, 305.722, 28.3574, 0.00110862, 305.722, 28.3574, 0.000186017, 266.981, 15.4055, 8.79192e-05, 264.918, 10.5915, 1.51546e-06, 295.521, 14.3739, 0.000163466, 305.722, -11.4153, 9.61615e-05, 303.773, 10.0184, 6.87079e-05, 266.408, 15.6347, 5.71158e-05, 300.22, 11.6231, 0.000222823, 303.773, 10.0184, 0.000104452, 303.773, 10.0184, 0.000102373, 305.722, 28.3574, 1.66973e-05, 303.773, 10.0184, 6.41129e-05, 303.773, 10.0184, 0.000217752, 305.722, 28.3574, -2.61212e-05, 305.722, 28.3574, -0.000113066, 305.722, 28.3574, -9.41443e-05, 305.722, 28.3574, -4.25809e-05, 305.722, 28.3574, -4.20301e-05, 305.722, 28.3574, -9.83043e-05, 295.521, 14.3739, 0.000115635, 295.521, 14.3739, 5.95604e-05, 303.773, 10.0184, 3.85442e-05, 300.106, 11.6231, 2.08479e-06, 265.147, 15.1762, 0.001534, 265.147, 15.2909, 0.000403659, 277.411, 17.6978, 1.16935e-05, 300.106, 11.6231, 7.19692e-05, 264.918, 10.5915, 5.12429e-07, 300.106, 11.6231, 0.000133711, 299.647, 12.6546, 2.08373e-05, 299.647, 12.6546, 4.69444e-05, 277.526, 18.9586, 2.09668e-05, 277.411, 18.9586, 1.21541e-07, 300.22, 11.6231, 0.00028328, 265.147, 15.2909, 0.000123125, 300.106, 11.6231, 0.00010285, 300.22, 10.3623, 3.57781e-05, 300.22, 11.6231, 0.000434258, 300.22, 11.6231, 3.84749e-05, 303.773, 10.0184, 0.000677053, 305.722, -11.4153, -9.93756e-05], "hand_left_keypoints_2d": [265.745, 95.4771, 0.0372974, 259.828, 92.7302, 0.0203715, 241.445, 102.027, 0.0195201, 242.713, 102.45, 0.0147881, 245.248, 113.226, 0.0065461, 241.868, 100.971, 0.0281603, 243.347, 102.239, 0.0218141, 243.558, 102.661, 0.0108514, 264.054, 101.816, 0.00705783, 242.501, 100.337, 0.0150333, 245.248, 101.816, 0.0132623, 262.575, 101.394, 0.00824927, 263.42, 101.816, 0.00623959, 242.29, 100.126, 0.0152979, 252.433, 100.548, 0.0135462, 262.364, 101.816, 0.0114025, 262.786, 102.027, 0.00902417, 242.079, 102.027, 0.0176943, 243.558, 103.507, 0.0163129, 255.813, 108.366, 0.0113746, 262.786, 102.239, 0.00920461], "hand_right_keypoints_2d": [271.598, 89.3304, 0.0360237, 275.332, 84.6631, 0.0459374, 273.232, 81.8627, 0.0216838, 293.068, 77.6621, 0.0282709, 297.268, 78.829, 0.0290635, 266.231, 96.0979, 0.0386137, 292.601, 78.829, 0.134475, 298.902, 77.8955, 0.368995, 304.502, 79.2957, 0.582487, 291.667, 84.8964, 0.0403245, 295.635, 82.7962, 0.118934, 302.869, 84.4297, 0.239958, 307.069, 84.8964, 0.530732, 289.334, 88.6303, 0.0811655, 296.801, 87.6968, 0.172233, 302.636, 88.6303, 0.315795, 306.836, 89.3304, 0.52872, 290.267, 93.7643, 0.104969, 298.435, 94.231, 0.225489, 302.169, 94.231, 0.428711, 305.903, 94.4644, 0.49], "pose_keypoints_3d": [], "face_keypoints_3d": [], "hand_left_keypoints_3d": [], "hand_right_keypoints_3d": []}]}

monacv commented 3 years ago

shouldn't actually be i instead of item[6]?

import os
import os.path as osp
import numpy as np
import pickle
import trimesh
import torch
import argparse
from generate_utils import get_order_obj, get_joints, get_param
import pdb

def parse_args():
    parser = argparse.ArgumentParser(description='Train an iCAN on HICO')

    parser.add_argument('--smplx_path', dest='smplx_path',
            help='Path to your SMPLX model',
            default='', type=str)

    parser.add_argument('--gender', dest='gender',
            help='Use gender neutral or gender specific SMPLX' + 'model',
            default='male',choices=['neutral', 'male', 'female'],
            type=str )

    parser.add_argument('--res', dest='res',
            help='Path to your SMPLify-X result',
            default='', type=str)

    parser.add_argument('--img_path', dest='img_path',
            help='Path to your image',
            default='', type=str)

    parser.add_argument('--save_obj_path', dest='save_obj_path',
            help='Path to save generated obj',
            default='', type=str)

    args = parser.parse_args()
    return args

args = parse_args()
list_hoi, order_obj_list, obj_para_dict = get_order_obj()
data = pickle.load(open('Data/Trainval_GT_HICO_with_idx.pkl', 'rb'), encoding='latin1')

for key in data.keys():
    img = os.path.join(args.img_path, 'HICO_train2015_%08d.jpg' % (key))
    for i, item in enumerate(data[key]):
        # The structure of item here:
        # item[0]: image id
        # item[1]: hoi
        # item[2]: human bounding box
        # item[3]: object bounding box
        # item[4]: useless
        # item[5]: alphapose
        # item[6]: openpose index, -1 means none
        #print('item[6]: ', item[6])
        if os.path.exists(os.path.join(args.res, 'results/HICO_train2015_%08d/%03d.pkl' % (key, item[6]))):
                print(os.path.join(args.res, 'results/HICO_train2015_%08d/%03d.pkl' % (key, item[6])))

        if os.path.exists(os.path.join(args.res, 'results/HICO_train2015_%08d/%03d.pkl' % (key, i))):
                print(os.path.join(args.res, 'results/HICO_train2015_%08d/%03d.pkl' % (key, i)))

        # if not os.path.exists(os.path.join(args.res, 'results/HICO_train2015_%08d/%03d.pkl' % (key, item[6]))):
        #     continue
        if not os.path.exists(os.path.join(args.res, 'results/HICO_train2015_%08d' % (key))):
                continue 
        if not os.path.exists(os.path.join(args.res, 'results/HICO_train2015_%08d/%03d.pkl' % (key, i))):
                continue

        # pdb.set_trace()
        # result = pickle.load(open(os.path.join(args.res, 'results/HICO_train2015_%08d/%03d.pkl' % (key, item[6])), 'rb'),encoding='latin1')
        result = pickle.load(open(os.path.join(args.res, 'results/HICO_train2015_%08d/%03d.pkl' % (key, i)), 'rb'),encoding='latin1')

        hbox = item[2]
        obox = item[3]
        hoi_class = item[1][0]
        obj_name = list_hoi[hoi_class][0]    

        # mesh       = os.path.join(args.res, 'meshes/HICO_train2015_%08d/%03d.obj' % (key, item[6]))
        mesh       = os.path.join(args.res, 'meshes/HICO_train2015_%08d/%03d.obj' % (key, i))

        htri       = trimesh.load_mesh(mesh)
        vertice    = np.array(htri.vertices,dtype=np.float32)

        joints = get_joints(args,torch.FloatTensor(torch.from_numpy(vertice.reshape(1,-1,3))))
        shoulder_len = np.linalg.norm(joints[16] - joints[17])

        radius = obj_para_dict[obj_name]['ratio'] * shoulder_len
        gamma_min = obj_para_dict[obj_name]['gamma_min']
        gamma_max = obj_para_dict[obj_name]['gamma_max']  

        otri, obj_vertexs = get_param(result, hbox, obox, htri, img, radius, gamma_min, gamma_max)      

        path = os.path.join(args.save_obj_path, 'HICO_train2015_%08d' % key)
        if (not os.path.exists(path)):
            os.makedirs(path)

        file_path = os.path.join(args.save_obj_path, 'HICO_train2015_%08d/object_%03d.pkl' % (key, i))
        f = open(file_path,'wb') 
        pickle.dump(obj_vertexs, f, protocol=2)

        tmp = otri.export(os.path.join(args.save_obj_path, 'HICO_train2015_%08d/object_%03d.obj' % (key, i)))
        alltri = otri + htri
        tmp = alltri.export(os.path.join(args.save_obj_path, 'HICO_train2015_%08d/human_object_%03d.obj' % (key, i)))

@DirtyHarryLYL @Foruck

Foruck commented 3 years ago

item[6] indicates which openpose output the pair is attached to. Therefore, the item[6] should not be replace by i. And the "person_id": [-1] in openpose output would not affect the results. Visualizing whether the openpose output is correctly mapped to the pair might help.