UttaranB127 / speech2affective_gestures

This is the official implementation of the paper "Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learning".
https://gamma.umd.edu/s2ag/
MIT License
44 stars 9 forks source link

Could not run the code. #7

Open Talegqz opened 2 years ago

Talegqz commented 2 years ago

Hi, thanks for your work. I am working on the gesture generation project, and I want to make some comparisons with your work. I followed your readme, but I could not run the code successfully, even I fixed too many bugs. Could you please review your code and make sure it can run following the readme?

zf223669 commented 2 years ago

How to fix the Bug? :( I have tried many days....

Talegqz commented 2 years ago

@zf223669 Sorry for the mistake in writing, and I want to say I could not run even fixed some Bugs. :( I have tried many days too.

UttaranB127 commented 2 years ago

Hi, re-pasting the response from issue #5, which I'm redirecting here:

It turns out that some of the packages were deprecated without compatibility since we released our code. As a result, preprocessing the data and later running training/inference needs you to have different versions of basic packages such as numpy installed. This requires strict versioning and modulation of our codebase that are beyond our scope at the moment. To circumnavigate the issue, I am uploading the preprocessed dataset in a single folder that you can download and directly use for training/inference. Please keep the entire contents of the download in the folder data/ted_db and point the variable data_path in main_v2.py to the data folder. I have also revised the code to make the data loading faster. Let me know if you still face issues.

zf223669 commented 2 years ago

Maybe you could rebuild the requirements.txt file with all the packages information so we could import the right version.

zf223669 commented 2 years ago

dear sir。 i got an error that could not find the file 0000000.npz when run the savezcompress function

UttaranB127 commented 2 years ago

Is your save path valid? savez_compressed is a standard numpy function, and it can complain about the path if the path is invalid (e.g., a parent directory in the path that does not exist).

zf223669 commented 2 years ago

Found the cache /home/zf223669/Mount/S2AG/data/ted_db/lmdb_test_s2ag_v2_cache_mfcc_14 building a language model... loaded from /home/zf223669/Mount/S2AG/data/ted_db/vocab_models_s2ag/vocab_cache.pkl Saved key 200037. Traceback (most recent call last): File "/home/zf223669/Mount/S2AG/speech2affective_gestures/main_v2.py", line 121, in train_data_ted, val_data_ted, test_data_ted = loader.load_ted_db_data(data_path, s2ag_config_args, args.train_s2ag, False) File "/home/zf223669/Mount/S2AG/speech2affective_gestures/loader_v2.py", line 614, in load_ted_db_data save_as_npz(train_dataset, 'val') File "/home/zf223669/Mount/S2AG/speech2affective_gestures/loader_v2.py", line 559, in save_as_npz mfcc_features=mfcc_features_np, aux_info=aux_info) File "<__array_function__ internals>", line 6, in savez_compressed File "/home/zf223669/Mount/miniconda3/envs/s2ag/lib/python3.7/site-packages/numpy/lib/npyio.py", line 689, in savez_compressed _savez(file, args, kwds, True) File "/home/zf223669/Mount/miniconda3/envs/s2ag/lib/python3.7/site-packages/numpy/lib/npyio.py", line 715, in _savez zipf = zipfile_factory(file, mode="w", compression=compression) File "/home/zf223669/Mount/miniconda3/envs/s2ag/lib/python3.7/site-packages/numpy/lib/npyio.py", line 112, in zipfile_factory return zipfile.ZipFile(file, *args, **kwargs) File "/home/zf223669/Mount/miniconda3/envs/s2ag/lib/python3.7/zipfile.py", line 1240, in init self.fp = io.open(file, filemode) FileNotFoundError: [Errno 2] No such file or directory: '/home/zf223669/Mount/S2AG/data/ted_db/individual/val/000000.npz'

UttaranB127 commented 2 years ago

This stack trace is not possible for me to follow without knowing your path structure. Please make sure all the paths are valid, or try some other workaround, e.g., skipping the keys that cause errors.

zf223669 commented 2 years ago

Hi, It could work now!I found that the save_cache() could not create the npz-train npz-test npz-val automatically, so I create the fould manually , commend the "if not os.path.exists(train_dir_nam/val_dir_name/test_dir_name) , "and it could generate the 00000.npz files..... it`s wired.

UttaranB127 commented 2 years ago

Great, I'm glad it works now! Thanks for detailing your steps, too!

Talegqz commented 2 years ago

Hi, I still could not run the code; when I fix the code as above, it still reports: Warning! No saved model was found at epoch 290. Speech to emotive gestures model not found I don't know how to fix it. I have downloaded the pre-trained model and put it on the ./models/ted_db And Could you please review your code and make sure it can run following the readme? I have cloned the code again in a clean environment and followed the README, but I cannot run it successfully. Thanks for your help!

Talegqz commented 2 years ago

I fix it by putting pre-trained model in ./models/s2ag_v2_mfcc_run_3 and moving

if len(all_models) < 2:

#     return '', None, np.inf

in processor_v2.py But I still hope that the author makes sure it can run following the readme, which will help will help those who want to follow your work.

zf223669 commented 2 years ago

I think a lot of bugs in this project are caused by incorrect paths and mismatched versions of dependencies.

Talegqz commented 2 years ago

I think a lot of bugs in this project are caused by incorrect paths and mismatched versions of dependencies. Yes, I fix them. BTW, did you know how to train Trinity Dataset in this project?

UttaranB127 commented 2 years ago

Thanks for your patience and feedback regarding the bugs, I will definitely revise the readme when I get some free time. Also, we did not train our model on the Trinity Dataset, only tested on it.