search

GalaxyCong / StyleDubber

[ACL 2024] This is the Pytorch code for our paper "StyleDubber: Towards Multi-Scale Style Learning for Movie Dubbing"
MIT License
50 stars 3 forks source link

Missing data folder and Google Drive data links #4

Open RishabhDahale opened 1 month ago

RishabhDahale commented 1 month ago

Hi there,

I loved the results, and was trying to replicate them. I'm trying to run the Setting3 inference script and have downloaded the checkpoints, but it's giving error that /data/conggaoxiang/dataset/Denoise_version2_all_feature/stats.json file is not present. Can someone please help resolve this? Also, can you provide the Google Drive data download links.

Thanks and regards, Rishabh

GalaxyCong commented 1 month ago

Hi,

thank you very much for your attention! ❤️

Solution🔧: Comment out lines 45-82 in style_dubber/modules.py.

Reason: There is no need to load the stats.json file because StyleDubber does not use pitch and energy for prediction. The stats.json file records the min and max values ​​of the dataset pitch and energy. image

By the way, if you need to obtain stats.json to improve StyleDubber, you can refer to the data preprocessing process.

We will upload it to Google Drive as soon as possible. maybe within a week.

RishabhDahale commented 1 month ago

Thanks for the response, but even after this fix, there are more issues popping up with the inference scripts. Now I'm getting the error

StyleDubber/MultimodalTransformer/src/models_16_5_Face2_same.py", line 68, in __init__ with open( FileNotFoundError: [Errno 2] No such file or directory: '/data/conggaoxiang/dataset/Denoise_version2_all_feature/speakers.json'

I'm trying to replicate the results and none of the inference scrips are working. Can you help me with these error

GalaxyCong commented 1 month ago

Thank you for your feedback on the issue. I'm glad to answer this question.

The key to solving the problem is to download the dataset. You could download the data before running code. Because the inference scrips need to load data from the dataset.

image

The download link for Google Drive should appear as soon as possible, not just Baidu Drive.

RishabhDahale commented 1 month ago

Thanks a lot. I'll wait for the google drive download link

GalaxyCong commented 1 month ago

Hi,

the dataset has been uploaded to Google Drive, welcome to test it!

RishabhDahale commented 1 month ago

Thanks a lot. Will test it out :-)