search

Gait3D / Gait3D-Benchmark

This is the code for the paper "Gait Recognition in the Wild with Dense 3D Representations and A Benchmark. (CVPR 2022)", "Gait Recognition in the Wild with Multi-hop Temporal Switch", and "Parsing is All You Need for Accurate Gait Recognition in the Wild".
133 stars 19 forks source link

Data pretreatment error in test SMPLGait #25

Closed faiimea closed 3 months ago

faiimea commented 3 months ago

I had some problems trying to reproduce SMPLgait. I processed the Data in the Data Pretreatment procedure. Since I was not clear what should be set for the dataset_root in the smplgait_gait3d.yaml, I set it as Gait3D-pkls/Gait3D-merged-pkl/ Then I try to run test.sh, and I get an error

[rank0]: Traceback (most recent call last):
[rank0]:   File "opengait/main.py", line 132, in <module>
[rank0]:     run_model(cfgs, training)
[rank0]:   File "opengait/main.py", line 60, in run_model
[rank0]:     Model.run_test(model)
[rank0]:   File "/home/lfz/workspace/Gait3D-Benchmark/opengait/modeling/base_model.py", line 446, in run_test
[rank0]:     info_dict = model.inference(rank)
[rank0]:   File "/home/lfz/workspace/Gait3D-Benchmark/opengait/modeling/base_model.py", line 384, in inference
[rank0]:     retval = self.forward(ipts)
[rank0]:   File "/home/lfz/workspace/Gait3D-Benchmark/opengait/modeling/models/smplgait.py", line 64, in forward
[rank0]:     outs = self.Backbone(sils)  # [n, c, s, h, w]
[rank0]:  The File "/ home/LFZ /. Conda/envs/py38torch1121 / lib/python3.8 / site - packages/torch/nn/modules/module. Py", line 1532, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:  The File "/ home/LFZ /. Conda/envs/py38torch1121 / lib/python3.8 / site - packages/torch/nn/modules/module. Py", line 1541, in _call_impl
[rank0]:     return forward_call(*args, **kwargs)
[rank0]:   File "/home/lfz/workspace/Gait3D-Benchmark/opengait/modeling/modules.py", line 46, in forward
[rank0]:     x = self.forward_block(x.transpose(
[rank0]: RuntimeError:  cannot reshape tensor of 0 elements into shape [-1, 1, 64,  0] because the unspecified dimension size -1 can be any value and is ambiguous

(Only one rank is shown) I locate the problem in the outs = self.Backbone(sils) in smplgait.py, so I print the sils at this time, the content is tensor([], device='cuda:0', size=(1, 1, 47, 64, 0)). I was confused about this, so I downloaded the dataset and did not do the python datasets/pretreatment.py, instead, I unpacked Gait3D-pkls.zip and Gait3D-parsing-64-64-pkl.zip and obtained Gait3D-merged-pkl/ Gait3D-parsing-64-64-pkl/ Gait3D-sils-128-88-pkl/ Gait3D-sils-64-44-pkl/ Gait3D-smpls-pkl/ I guessed that this was equivalent to Pretreatment, while no error was reported while running the merge Therefore, I don't know where the data went wrong, which led to this error. Hope you can answer this question

In addition, I have a problem. Because the current merge_three_modality requires me to provide the parsing address, but I need to reproduce the smplgait method of Gait3D data set, so the parsing data set is not required. However, I don't know why dataset_root in config should be set I tried a few things, but I got an error, and when I set it to Gait3D-pkls, I got an error

[rank0]: Traceback (most recent call last):
[rank0]:   File "opengait/main.py", line 132, in <module>
[rank0]:     run_model(cfgs, training)
[rank0]:   File "opengait/main.py", line 60, in run_model
[rank0]:     Model.run_test(model)
[rank0]:   File "/home/lfz/workspace/Gait3D-Benchmark/opengait/modeling/base_model.py", line 467, in run_test
[rank0]:     return eval_func(info_dict, dataset_name, **valid_args)
[rank0]:   File "/home/lfz/workspace/Gait3D-Benchmark/opengait/evaluation/evaluator.py", line 247, in evaluate_Gait3D
[rank0]:     features, labels, cams, time_seqs = data['embeddings'], data['labels'], data['types'], data['views']
[rank0]: KeyError: 'embeddings'

I used the Model-64 in the model SMPLGait(CVPR2022) If I did something wrong, please feel free to tell me

JinkaiZheng commented 3 months ago

Hi~ Please do not use Gait3D-sils-64-44-pkl.

First, run the following command to obtain Gait3D-sils-64-64-pkl: python datasets/pretreatment.py -i 'Gait3D/2D_Silhouettes' -o 'Gait3D-sils-64-64-pkl' -r 64

Next, run this command to get Gait3D-merged-pkl: python datasets/Gait3D/merge_three_modality.py --pars_path 'Gait3D-pars-64-64-pkl' --sils_path 'Gait3D-sils-64-64-pkl' --smpls_path 'Gait3D-smpls-pkl' --output_path 'Gait3D-merged-pkl' --link 'hard'

At last, set Gait3D-merged-pkl in the dataset_root in the smplgait_gait3d.yaml.

JinkaiZheng commented 3 months ago

The Gait3D-sils-64-44-pkl is used in our old code version. We are so sorry that we did not update it in the data packet in time.

faiimea commented 3 months ago

Thanks, now I have successfully tested the model.

I have another problem. Although I am using the Gait3d-merged data set that includes Gait3D-parsing-64-64-pkl, the model I am using is the SMPLGait (CVPR2022) model in modelzoo for the Gait3D data set. parsing is irrelevant, that is, if I need to test the model with a new data set, I just need to provide 2D silhouette and 3Dsmpl parameters, but not parsing data？

JinkaiZheng commented 3 months ago

The data_in_use in smplgait_gait3d.yaml declares which data to use. From left to right, it is parsing, silhouette, and smpl.

faiimea commented 3 months ago

Cool. I believe it solves my problem. Thanks :)