yerfor / GeneFacePlusPlus

GeneFace++: Generalized and Stable Real-Time 3D Talking Face Generation; Official Code
MIT License
1.59k stars 229 forks source link

Missing checkpoints #230

Open nitinmukesh opened 2 months ago

nitinmukesh commented 2 months ago

Hello,

I managed to install on Windows and placed checkpoints as mentioned in readme

Step 2: Download pre-processed dataset of May([Google Drive](https://drive.google.com/drive/folders/1SwZ7uRa5ESzzq_Cd21-Lk5heAZxa9oZO?usp=sharing) or [BaiduYun Disk](https://pan.baidu.com/s/1U_FalVoxgb9sAb9FD1cZEw?pwd=98n4) with password 98n4), and place it here data/binary/videos/May/trainval_dataset.npy

Step 3: Download pre-trained audio-to-motino model audio2motion_vae.zip ([Google Drive](https://drive.google.com/drive/folders/1M6CQH52lG_yZj7oCMaepn3Qsvb-8W2pT?usp=sharing) or [BaiduYun Disk](https://pan.baidu.com/s/19UZxMrO-ZvkOeYzUkOKsTQ?pwd=9cqp) with password 9cqp) and motion-to-video checkpoint motion2video_nerf.zip, which is specific to May (in this [Google Drive](https://drive.google.com/drive/folders/1M6CQH52lG_yZj7oCMaepn3Qsvb-8W2pT?usp=sharing) or in this[BaiduYun Disk](https://pan.baidu.com/s/1U_FalVoxgb9sAb9FD1cZEw?pwd=98n4) with password 98n4), and unzip them to ./checkpoints/

Still getting this error

(C:\tut\GeneFacePlusPlus\GFPP) C:\tut\GeneFacePlusPlus>python inference/app_genefacepp.py --a2m_ckpt=checkpoints/audio2motion_vae --head_ckpt= --torso_ckpt=checkpoints/motion2video_nerf/may_torso
| WARN: egs/egs_bases/audio2motion/vae.yaml not exist.
| WARN: checkpoints\th1kh_512_audio2motion\base.yaml not exist.
| Hparams:  {
  "accumulate_grad_batches": 1,
  "amp": false,
  "audio_type": "hubert",
  "base_config": [
    "egs/egs_bases/audio2motion/vae.yaml",
    "../th1kh_512_audio2motion/base.yaml"
  ],
  "batch_size": 4,
  "binarization_args": {
    "with_coeff": true,
    "with_hubert": true,
    "with_mel": true
  },
  "binary_data_dir": "data/binary/voxceleb2_audio2motion",
  "blink_mode": "blink_unit",
  "clip_grad_norm": 1,
  "clip_grad_value": 0,
  "debug": false,
  "ds_name": "TH1KH_512",
  "eval_max_batches": 10,
  "exp_name": "",
  "gen_dir_name": "",
  "hidden_size": 256,
  "infer": false,
  "infer_audio_source_name": "",
  "infer_ckpt_steps": 40000,
  "infer_out_npy_name": "",
  "init_from_ckpt": "",
  "init_method": "tcp",
  "lambda_kl": 0.02,
  "lambda_kl_t1": 2000,
  "lambda_kl_t2": 2000,
  "lambda_l2_reg_exp": 0.1,
  "lambda_mse_exp": 1.0,
  "lambda_mse_lm2d": 0.0,
  "lambda_mse_lm3d": 0.0,
  "load_ckpt": "",
  "load_db_to_memory": false,
  "lr": 0.0005,
  "max_sentences_per_batch": 512,
  "max_tokens_per_batch": 20000,
  "max_updates": 400000,
  "motion_type": "exp",
  "num_ckpt_keep": 100,
  "num_sanity_val_steps": 5,
  "num_valid_plots": 1,
  "num_workers": 4,
  "optimizer_adam_beta1": 0.9,
  "optimizer_adam_beta2": 0.999,
  "print_nan_grads": false,
  "process_id": 0,
  "raw_data_dir": "/home/tiger/datasets/raw/TH1KH_512",
  "ref_id_mode": "first_frame",
  "resume_from_checkpoint": 0,
  "sample_min_length": 32,
  "save_best": false,
  "save_codes": [
    "tasks",
    "modules",
    "egs"
  ],
  "save_gt": true,
  "scheduler": "exponential",
  "seed": 9999,
  "smo_win_size": 5,
  "split_seed": 999,
  "start_rank": 0,
  "syncnet_ckpt_dir": "checkpoints/0904_syncnet/syncnet_hubert_vox2",
  "task_cls": "tasks.os_avatar.audio2secc_task.Audio2SECCTask",
  "tb_log_interval": 100,
  "total_process": 1,
  "use_eye_amp_embed": false,
  "use_flow": true,
  "use_fork": true,
  "use_kv_dataset": true,
  "use_mouth_amp_embed": true,
  "use_pitch": true,
  "val_check_interval": 2000,
  "valid_infer_interval": 2000,
  "valid_monitor_key": "val_loss",
  "valid_monitor_mode": "min",
  "validate": false,
  "warmup_updates": 1000,
  "weight_decay": 0,
  "work_dir": "",
  "world_size": -1
}
| load 'model' from 'checkpoints/audio2motion_vae\model_ckpt_steps_400000.ckpt', strict=True
| WARN: checkpoints\motion2video_nerf\may_torso\lm3d_radnerf_torso.yaml not exist.
| load 'model' from 'checkpoints/motion2video_nerf/may_torso\model_ckpt_steps_250000.ckpt', strict=True
trainval: Smooth head trajectory (rotation and translation) with a window size of 7
Error in sys.excepthook:

Original exception was:

From where should I download these

WARN: egs/egs_bases/audio2motion/vae.yaml not exist. WARN: checkpoints\th1kh_512_audio2motion\base.yaml not exist. WARN: checkpoints\motion2video_nerf\may_torso\lm3d_radnerf_torso.yaml not exist.