Error running the data prep script

[oscarchanggoogle]

First, I downloaded lrs3_pretrain.zip, lrs3_test_v0.4.zip, and lrs_3_v0.4_txt.zip, and made sure the checksums matched. Unzipping them gave me three folders: pretrain, lrs3_v0.4, and test. I copied out lrs3_v0.4/trainval and placed it in the root folder beside pretrain.

Next, I ran the command: python3 get_data.py --root-path . --src-lang en

I got an error with "Creating AVSR manifests for en" KeyError: 'iW4fCwfw1vg/00033'

Can you please let me know I'm doing wrong?

[Anwarvic]

Hi @oscarchanggoogle,

Thanks for raising this issue!

Could you post the whole error trace so we can have a better understanding of what could've gone wrong?

[oscarchanggoogle]

Thanks for looking into this @Anwarvic . I've uploaded an image of my terminal.

[Anwarvic]

Hi @oscarchanggoogle

Thank you for posting the full error trace! From the uploaded image, you can see that the trainval set hasn't been processed (attached below for your reference):

Please, make sure that the /usr/local/google/home/oscarchang/Downloads/muavic/lrs3/trainval directory contains both txt files (*.txt) and videos (*.mp4).

My guess is that you didn't download the trainval data, you only downloaded the text lrs_3_v0.4_txt.zip. If that is the case, please download the set the same way you did with pretrain.zip, uncompress it inside /usr/local/google/home/oscarchang/Downloads/muavic/lrs3 directory, and re-run the get_data.py script as you did earlier. The script should ignore all already-processed files and process only the new ones.

Hope this helps!

[oscarchanggoogle]

Whoops you are right, for some reason, my previous directory of trainval only had the txt. Now I re-downloaded it and it has the mp4.

But I get the same error. I've attached another screenshot. I've checked that trainval/iW4fCwfw1vg/00018 does not actually exist, only trainval/iW4fCwfw1vg/50018.txt and trainval/iW4fCwfw1vg/50018.mp4.

[Anwarvic]

Hi @oscarchanggoogle ,

Sorry to hear that you still having issues!

This error indicates that there is an audio file with the name 00018.wav inside ~/Downloads/muavic/muavic/en/audio/train/iW4fCwfw1vg directory. Could you please check if this audio file exists?

If yes, then something went wrong in the segmentation. The original file located in ~/Downloads/muavic/lrs3/pretrain/iW4fCwfw1vg/00018.mp4 should be segmented into ~/Downloads/muavic/seg_pretrain/iW4fCwfw1vg/00018_{0,1,2,3,4}.mp4. And then later the audio is extracted and written to ~/Downloads/muavic/muavic/en/audio/train/iW4fCwfw1vg/00018_{0,1,2,3,4}.wav.

So, I would suggest delete the audio file ~/Downloads/muavic/muavic/en/audio/train/iW4fCwfw1vg/00018.wav and the video file ~/Downloads/muavic/muavic/en/video/train/iW4fCwfw1vg/00018.mp4 and re-run the script.

Due to the nature of your issue, it is so hard to replicate. So, probably we are going to go back and forth multiple times in order to resolve this issue. So, kindly plan accordingly!

[oscarchanggoogle]

Thanks! That fixed my error. I got the script fully running for en!

For the other languages, I have warnings/errors that look like the following: "/usr/local/google/home/oscarchang/Downloads/muavic/mtedx_utils.py:207: UserWarning: TED talk 4lJMAP0UVU8 hasn't been downloaded... skipping!!"

Is this expected? Or should I re-run the script?

[Anwarvic]

Glad that everything is working now!

Regarding your question, yes... this is expected! This message just warns you that some of the TED talks (in your case 4lJMAP0UVU8) hasn't been downloaded from YouTube. This could be for multiple reasons; for example the talk is private, or it has been removed, or the downloading script wasn't able to download it. A full list of failed-to-download talks be found here: /usr/local/google/home/oscarchang/Downloads/muavic/mtedx/not_found_videos.txt

If you don't mind, I'm gonna close this issue now!