How to preprocess dataset?

xljh0520 commented 2 years ago

Hi, I noticed that your facefore_dataset.py contains these code. I downloaded lrw dataset and unzip them. I only got a folder named lipread_mp4. What should I do to achieve these file such as .npy?

video_path = os.path.join(paths[0] + '_crop.mp4')
lmark_path = os.path.join(paths[0]+ '_original.npy') 
ani_path = os.path.join(paths[0]+"_ani.mp4")
rt_path = os.path.join(paths[0]+ '_rt.npy') 
front_path = os.path.join(paths[0]+ '_front.npy')

xljh0520 commented 2 years ago

Btw, I found another problem when I tried to train with voxceleb2 dataset.

This is my train scripts. And I only change the dataroot.

train_vox_new_nonlinear(){
    CUDA_VISIBLE_DEVICES=2,3 python train.py \
    --name face8_vox_new \
    --dataset_mode facefore \
    --adaptive_spade \
    --warp_ref \
    --warp_ani \
    --spade_combine \
    --add_raw_loss \
    --gpu_ids 0,1 \
    --batchSize 4 \
    --nThreads 8 \
    --niter 1000 \
    --niter_single 1001 \
    --n_shot 8 \
    --n_frames_G 1 \
    --dataroot '/home/wei/exp/vox/vox/mp4' \
    --dataset_name vox \
    --save_epoch_freq 1 \
    --display_freq 5000 \
    --continue_train \
    --use_new \
    --crop_ref
}

train_vox_new_nonlinear

However in line 725 of facefore_dataset.py, I met a problem that this is no folder named pickle in my downloaded vox dataset. What should I do to preprocess the dataset?

if opt.isTrain:
    _file = open(os.path.join(self.root, 'pickle','dev_lmark2img.pkl'), "rb")
    self.data = pkl.load(_file)
    _file.close()

xiaoyun4 commented 2 years ago

Hi @xljh0520，I also met the problem that this is no folder named pickle including dev_lmark2img.pkl. How do you solve this issue?

xljh0520 commented 2 years ago

Hi @xljh0520，I also met the problem that this is no folder named pickle including dev_lmark2img.pkl. How do you solve this issue?

I think we should follow the readme doc in data folder to get these file. However, I met another problem when propressing a video. The problem is described in another issue. Hope it helps you well.

lelechen63 / Talking-head-Generation-with-Rhythmic-Head-Motion

How to preprocess dataset? #12