RuntimeError: Motion synthesis using conditional model

[Lee296]

Hello,

Thanks for your work. I have encountered a runtime ValueError while using the conditional model when checking the synthesis result through either using text promot or keyframe locations. The command and error are listed as follow.

I found that syntiesis using unconditioned model can work fine. (P.s I think the correct command for unconditioned model shold be sample.synthesize instead of sample.synthesis in the README)

I 'm curious as to what might cause this issue.

Thank you.

python -m sample.conditional_synthesis --model_path ./save/condmdi_randomframes/model000750000.pt --edit_mode uncond --num_samples 10 --num_repetitions 3

python -m sample.conditional_synthesis --model_path ./save/condmdi_randomframes/model000750000.pt --edit_mode benchmark_sparse --transition_length 5 --num_samples 10 --num_repetitions 3 --text_prompt "a person throws a ball"

both return error:

Traceback (most recent call last):
  File "/home/anaconda3/envs/sig24ibt/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/anaconda3/envs/sig24ibt/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/Desktop/PyProject/diffusion-motion-inbetweening/sample/conditional_synthesis.py", line 314, in <module>
    main()
  File "/home/Desktop/PyProject/diffusion-motion-inbetweening/sample/conditional_synthesis.py", line 127, in main
    data = load_dataset(args, max_frames, split=split)
  File "/home/Desktop/PyProject/diffusion-motion-inbetweening/sample/conditional_synthesis.py", line 309, in load_dataset
    data = get_dataset_loader(conf)
  File "/home/Desktop/PyProject/diffusion-motion-inbetweening/data_loaders/get_data.py", line 78, in get_dataset_loader
    dataset = get_dataset(conf)
  File "/home/Desktop/PyProject/diffusion-motion-inbetweening/data_loaders/get_data.py", line 68, in get_dataset
    drop_redundant=conf.drop_redundant)
  File "/home/Desktop/PyProject/diffusion-motion-inbetweening/data_loaders/humanml/data/dataset.py", line 1171, in __init__
    drop_redundant=self.drop_redundant,)
  File "/home/Desktop/PyProject/diffusion-motion-inbetweening/data_loaders/humanml/data/dataset.py", line 340, in __init__
    *sorted(zip(new_name_list, length_list), key=lambda x: x[1]))
ValueError: not enough values to unpack (expected 2, got 0)

[jackson09255921]

Hey, @Lee296 You can inspect that there is no condmdi_randomframes directory in save directory, but condmdi_random_frames. Thus, you should change ./save/condmdi_randomframes/model000750000.pt to ./save/condmdi_random_frames/model000750000.pt

[setarehc]

Hi @Lee296,

Thanks for your interest in our work.

Thanks for pointing out the typo. The correct script name for unconditioned synthesis is sample/synthesize.py. I fixed the README with the correct name.

Your error is due to the data not being loaded correctly. For unconditioned synthesis, there is no reference sample to be loaded and thus sample.synthesize works for you. But for conditioned synthesis (and editing), you need reference samples for which the test set will be loaded.

Can you check if you've followed the dataset preparation steps according to the README and update me?

Best, Setareh

[od946]

Hi @setarehc ,

Indeed, it seems that there's an issue with the code/data. I followed the instructions, renamed the folder above as @jackson09255921 pointed out, and the error persists.

Thank you for assistance!

[setarehc]

In dataset/ you should have the HumanML3D folder containing the original motion representations (3D joint positions) called new_joints_abs_3d which must contain 29228 files as wells the processed motion representations called new_joint_vecs_abs_3d which mush also contain 29228 files.

Can you let me know of the contents of your dataset folder and the HumanML3D folder inside it?

Best, Setareh

[od946]

No, I don't have these subfolders under HumanML3D. I followed these instructions from the readme, should have I done something else?

cd ..
git clone https://github.com/EricGuo5513/HumanML3D.git
unzip ./HumanML3D/HumanML3D/texts.zip -d ./HumanML3D/HumanML3D/
cp -r HumanML3D/HumanML3D diffusion-motion-inbetweening/dataset/HumanML3D
cd diffusion-motion-inbetweening
cp -a dataset/HumanML3D_abs/. dataset/HumanML3D/

[setarehc]

Seems like you have followed the instructions for Generation only which only includes the text descriptios and not the motions and lets you do uncoditioned sampling. As a result, you cannot do anything that needs the motion sequences, including training, conditional sampling or editing. Please follow the instructions for Full data aquisition and you should be all set.

Best, Setareh