Failed to creating the 3D avatar

[rakeshparihar]

@jpcurbelo These data file are also missing, where can i get the data files?

human-body-reshape-DL-paper/measurement/lib/python3.10/site-packages/numpy/core/shape_base.py", line 296, in vstack return _nx.concatenate(arrs, 0, dtype=dtype, casting=casting) File "<__array_function__ internals>", line 200, in concatenate ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 3 dimension(s) and the array at index 1 has 2 dimension(s)

/data/body_reshaper_files/rfemats_male.npy
/data/body_reshaper_files/rfemats_female.npy
/data/body_reshaper_files/rfemasks_male.npy
/data/body_reshaper_files/rfemasks_female.npy
datasets/measurements_TOTAL_male.npy

[jpcurbelo]

Hi,@rakeshparihar. I just committed the files you are requesting. Please, let me know if you have access to them and if is there anything else left.

All the files needed to generate the 3D avatars are to be generated by training the reshaper model (src/reshaper/trainer.py). Although this might take a few hours, I recommend going through it to become familiar with the steps. Also, the measurements_TOTAL_gender.npy files are generated by running src/datasets/ds_measurements_filter.py after training the reshaper.

I will try to make this more clear in the Readme file anyways.

Thanks for trying!

[rakeshparihar]

@jpcurbelo thanks for share the training data here , also struggling with

[4] Starting to create the 3D avatar.
Wrong file or file path --> '/Users/reshape_humanbody/scr/../data/body_reshaper_files/def2vert_male.npz' file not found.
Traceback (most recent call last):
  File "/Users/reshape_humanbody/scr/photos2avatar.py", line 590, in <module>
    main()

Can you also update these file for initial setup ?

[rakeshparihar]

[3] starting to load vertices from .obj files for female
>>  converting female body 1986
[3] finished loading vertices from .obj files for female in 43.550944s
(3, 1986) 1986
>> calculating weights of female body 1986

[4] starting to measure bodies from .obj data for female
>> calculating measurements of female body 1Traceback (most recent call last):
  File "/Users/reshape_humanbody/scr/reshaper/trainer.py", line 742, in <module>
    train()
  File "/Users/reshape_humanbody/scr/reshaper/trainer.py", line 699, in train
    measurements = measure_bodies(cp, vertices, vol, label = gender)
  File "/Users/reshape_humanbody/scr/reshaper/trainer.py", line 236, in measure_bodies
    measurements[:, i] = calc_measurements(cp, vertices[i, :, :], vol[i]).flat
  File "/Users/reshape_humanbody/scr/reshaper/trainer.py", line 279, in calc_measurements
    return np.array(measurement_list, dtype=np.float64).reshape(M_NUM, 1)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (21,) + inhomogeneous part.

//------

[4] Starting to create the 3D avatar.
Wrong file or file path --> '/Users/reshape_humanbody/scr/../data/body_reshaper_files/cp_male.pkl' file not found.
Traceback (most recent call last):
  File "/Users/reshape_humanbody/scr/photos2avatar.py", line 590, in <module>
    main()
  File "/Users/reshape_humanbody/scr/photos2avatar.py", line 583, in main
    _ = avatar.measure(out_meas_name=f"output_data_avatar_{input_gender_glob}_fromImg")
  File "/Users/reshape_humanbody/scr/reshaper/avatar.py", line 347, in measure
    self.output_data = self.calc_measurements(cp, facets)

[jpcurbelo]

@jpcurbelo thanks for share the training data here , also struggling with

[4] Starting to create the 3D avatar.
Wrong file or file path --> '/Users/reshape_humanbody/scr/../data/body_reshaper_files/def2vert_male.npz' file not found.
Traceback (most recent call last):
  File "/Users/reshape_humanbody/scr/photos2avatar.py", line 590, in <module>
    main()

Can you also update these file for initial setup ?

Just committed the def2vert files. I believe those were the only ones missing - the rest are created during the training but not used to build the 3D avatar.

[jpcurbelo]

[3] starting to load vertices from .obj files for female
>>  converting female body 1986
[3] finished loading vertices from .obj files for female in 43.550944s
(3, 1986) 1986
>> calculating weights of female body 1986

[4] starting to measure bodies from .obj data for female
>> calculating measurements of female body 1Traceback (most recent call last):
  File "/Users/reshape_humanbody/scr/reshaper/trainer.py", line 742, in <module>
    train()
  File "/Users/reshape_humanbody/scr/reshaper/trainer.py", line 699, in train
    measurements = measure_bodies(cp, vertices, vol, label = gender)
  File "/Users/reshape_humanbody/scr/reshaper/trainer.py", line 236, in measure_bodies
    measurements[:, i] = calc_measurements(cp, vertices[i, :, :], vol[i]).flat
  File "/Users/reshape_humanbody/scr/reshaper/trainer.py", line 279, in calc_measurements
    return np.array(measurement_list, dtype=np.float64).reshape(M_NUM, 1)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (21,) + inhomogeneous part.

//------

[4] Starting to create the 3D avatar.
Wrong file or file path --> '/Users/reshape_humanbody/scr/../data/body_reshaper_files/cp_male.pkl' file not found.
Traceback (most recent call last):
  File "/Users/reshape_humanbody/scr/photos2avatar.py", line 590, in <module>
    main()
  File "/Users/reshape_humanbody/scr/photos2avatar.py", line 583, in main
    _ = avatar.measure(out_meas_name=f"output_data_avatar_{input_gender_glob}_fromImg")
  File "/Users/reshape_humanbody/scr/reshaper/avatar.py", line 347, in measure
    self.output_data = self.calc_measurements(cp, facets)

Yes, I was able to reproduce that one here. Something wrong with some shape - will take a look later. However, if one re-runs the training, then the previous files are loaded and the error does not show up. Please, try and let me know.

[rakeshparihar]

@jpcurbelo (reshape) user Reshaper % python trainer.py
gender = female [4] starting to measure bodies from .obj data for female

calculating measurements of female body 1986 [4] finished measuring bodies from .obj data for female in 61.563182s gender = male [3] starting to load vertices from .obj files for male converting male body 4082 [3] finished loading vertices from .obj files for male in 113.954148s (3, 4082) 4082 calculating weights of male body 4082

[4] starting to measure bodies from .obj data for male

calculating measurements of male body 1Traceback (most recent call last): File "/Users/reshape_humanbody/scr/reshaper/trainer.py", line 742, in train() File "/Users/reshape_humanbody/scr/reshaper/trainer.py", line 699, in train measurements = measure_bodies(cp, vertices, vol, label = gender) File "/Users/reshape_humanbody/scr/reshaper/trainer.py", line 236, in measure_bodies measurements[:, i] = calc_measurements(cp, vertices[i, :, :], vol[i]).flat File "/Users/reshape_humanbody/scr/reshaper/trainer.py", line 279, in calc_measurements return np.array(measurement_list, dtype=np.float64).reshape(M_NUM, 1) ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (21,) + inhomogeneous part.

//--------

Please install GPU version of TF

[Errno 17] File exists: '/Users/reshape_humanbody/scr/../data/model_files'

df_total.shape = (600, 10) Traceback (most recent call last): File "/Users/reshape_humanbody/scr/extractor/extractor_model_training.py", line 716, in main() File "/Users/reshape_humanbody/scr/extractor/extractor_model_training.py", line 70, in main imgX_front, imgX_side = load_images() File "/Users/reshape_humanbody/scr/extractor/extractor_model_training.py", line 304, in load_images open( FileNotFoundError: [Errno 2] No such file or directory: '/Users/reshape_humanbody/scr/../data/datasets/silhouettes_blender224_npy/silhouettes_SPRING_bw/silh_Xarray224_SPRING_female_bw_100test.npz'

//-------------------

cp_female.pkl cp_male.pkl are also missing

[jpcurbelo]

A couple of comments: 1- Not sure why you are training the reshaper on the ANSURII data (1986 female and 4082 male). Those files were generated by using the trained model. So, for this stage, you should use the SPRING data (1531 female and 1517 male)

def obj2npy(label = "female"): 
    """
    ...
    """
    print('[3] starting to load vertices from .obj files for %s'%(label))
    start = time.time()    
    obj_file_dir = os.path.join(OBJ_FILES_SPRING, label)
    file_list = sorted(os.listdir(obj_file_dir))

2- The cp_gender.pkl files were just uploaded. Although, they should have been created in the first step during the training. Not sure what this was not the case.

## generate and load control point from txt to pkl file
cp_gender_file = os.path.join(RESHAPER_FILES_DIR, f'cp_{gender}.pkl')
if not os.path.exists(cp_gender_file):
    cp = convert_cp(label = gender)
else:
    cp = np.load(open(cp_gender_file, "rb"), allow_pickle=True)

3- I'll try to solve this later ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (21,) + inhomogeneous part. However, you might continue the training by re-running the code. For some reason all works ok when the previous files are loaded instead of created.