search

JeremyCJM / DiffSHEG

[CVPR'24] DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generation
https://jeremycjm.github.io/proj/DiffSHEG/
BSD 3-Clause "New" or "Revised" License
112 stars 9 forks source link

About audio_feat_path #5

Closed h6kplus closed 3 months ago

h6kplus commented 3 months ago

Hi, nice work!

When I was reproducing your code, I got the following error: lmdb.Error: /ceph/data/talking_human/BEAT/beat_cache/beat_4english_15_141/train/aud_feat_cache/hubert_large_ls960_ft: No such file or directory

It seems that I need to store the encoded hubert features into this directory but I can't find any preprocessing code in the BEAT dataset to do that. Is there any suggestions on how to obtain the features?

JeremyCJM commented 3 months ago

Hi Yidong, thanks for your interest!

You can refer to the following function to compute the HuBERT feature, as mentioned in this issue https://github.com/JeremyCJM/DiffSHEG/issues/2#issuecomment-2095831017.

https://github.com/JeremyCJM/DiffSHEG/blob/3ebf3058f48cba3da9146afb7623e9ec1ab9e9a5/trainers/ddpm_beat_trainer.py#L1430

h6kplus commented 3 months ago

That works, thank you!