Open avia8abhi opened 1 year ago
Hi, can you provide more details about the error so that I can help you out?
It executes till step 8 (calculate audio features) with few warnings and errors but fails at step 9, Here's the exact error I am getting after running # 9. Calculate 3DMM given in process_data.sh.
(geneface) root@C.6305573:~/GeneFace-1.1.0$ python data_gen/nerf/extract_3dmm.py --video_id=$VIDEO_ID loading the model from deep_3drecon/checkpoints/facerecon/epoch_20.pth loading video ... extracting 2D facial landmarks ...: 100%|██████████████████████████████████████████████| 6073/6073 [03:34<00:00, 28.35it/s] start extracting 3DMM...: 0%| | 0/189 [00:00<?, ?it/s] Traceback (most recent call last):
File "/root/GeneFace-1.1.0/data_gen/nerf/extract_3dmm.py", line 112, in
Would appreciate if you could give your suggestions and solutions for getting through the errors by today, since it's been a week i am not able to proceed further.
hi, I have not confronted with this error, could you please print the w0, h0, s, t[0], t[1] as raised by:
File "/root/GeneFace-1.1.0/deep_3drecon/util/preprocess.py", line 205, in align_img trans_params = np.array([w0, h0, s, t[0], t[1]], dtype=np.float32)
I was able to fix the errors with trans_params = np.array([w0, h0, s, t[0], t[1]], dtype=np.float32), now in the next step i am getting this error where it says it couldn't found aud_deepspeech.npy..............
here's the exact error:
(geneface) root@C.6305573:~/GeneFace-1.1.0$ python data_gen/nerf/binarizer.py --config=egs/datasets/videos/${VIDEO_ID}/lm3d_radnerf.yaml
| Unknow hparams: []
| Hparams chains: ['egs/egs_bases/radnerf/base.yaml', 'egs/egs_bases/radnerf/lm3d_radnerf.yaml', 'egs/datasets/videos/May/lm3d_radnerf.yaml']
| Hparams:
accumulate_grad_batches: 1, ambient_out_dim: 2, amp: True, base_config: ['egs/egs_bases/radnerf/lm3d_radnerf.yaml'], binary_data_dir: data/binary/videos,
bound: 1, camera_offset: [0, 0, 0], camera_scale: 4.0, clip_grad_norm: 0, clip_grad_value: 0,
cond_out_dim: 64, cond_type: idexp_lm3d_normalized, cond_win_size: 1, cuda_ray: True, debug: False,density_thresh: 10, density_thresh_torso: 0.01, desired_resolution: 2048, dt_gamma: 0.00390625, eval_max_batches: 100,
exp_name: , far: 0.9, finetune_lips: True, finetune_lips_start_iter: 200000, geo_feat_dim: 128,
grid_interpolation_type: linear, grid_size: 128, grid_type: tiledgrid, gui_fovy: 21.24, gui_h: 512,
gui_max_spp: 1, gui_radius: 3.35, gui_w: 512, hidden_dim_ambient: 128, hidden_dim_color: 128,
hidden_dim_sigma: 128, individual_embedding_dim: 4, individual_embedding_num: 13000, infer: False, infer_audio_source_name: ,
infer_bg_img_fname: , infer_c2w_name: , infer_cond_name: , infer_lm3d_clamp_std: 2.5, infer_lm3d_lle_percent: 0.0,
infer_lm3d_smooth_sigma: 0.0, infer_out_video_name: , infer_scale_factor: 1.0, infer_smo_std: 0.0, infer_smooth_camera_path: True,
infer_smooth_camera_path_kernel_size: 7, lambda_ambient: 0.1, lambda_lpips_loss: 0.01, lambda_weights_entropy: 0.0001, load_ckpt: ,load_imgs_to_memory: False, log2_hashmap_size: 16, lr: 0.0005, max_ray_batch: 4096, max_steps: 16,
max_updates: 250000, min_near: 0.05, n_rays: 65536, near: 0.3, num_ckpt_keep: 1,num_layers_ambient: 3, num_layers_color: 2, num_layers_sigma: 3, num_sanity_val_steps: 2, num_steps: 16,
num_valid_plots: 5, optimizer_adam_beta1: 0.9, optimizer_adam_beta2: 0.999, print_nan_grads: False, processed_data_dir: data/processed/videos,
raw_data_dir: data/raw/videos, resume_from_checkpoint: 0, save_best: True, save_codes: ['tasks', 'modules', 'egs'], save_gt: True,scheduler: exponential, seed: 9999, smo_win_size: 5, smooth_lips: False, task_cls: tasks.radnerfs.radnerf.RADNeRFTask,
tb_log_interval: 100, torso_head_aware: False, torso_individual_embedding_dim: 8, torso_shrink: 0.8, update_extra_interval: 16,
upsample_steps: 0, use_window_cond: True, val_check_interval: 2000, valid_infer_interval: 10000, valid_monitor_key: val_loss,valid_monitor_mode: min, validate: False, video_id: May, warmup_updates: 0, weight_decay: 0,
with_att: True, work_dir: ,loading deepspeech ...
Traceback (most recent call last):
File "/root/GeneFace-1.1.0/data_gen/nerf/binarizer.py", line 277, in
Hi, it seems to be caused by the downloading of DeepSpeech ckpt was failed, so the process.sh just skip extracting the deepspeech. Maybe you could run this commandline to rerun the extracting deepspeech step, so the code will automatically download the deepspeech ckpt.
conda activate geneface
CUDA_VISIBLE_DEVICES=0 python data_util/process.py --video_id=May --task=2
Apologize for the late reply since I have been engaged in a new project. We plan to upload the code of GeneFace++ in Augest, and the reported bugs will be fixed then.
No issues I can understand, glad to hear that Geneface++ would also be out! Step 3 got executed properly, now facing a few errors while executing the final command bash scripts/infer_lm3d_radnerf.sh in step 4. Posting the errors in a few hours...many thanks for your support!
can you also post the error log here?
I have been facing a few errors while executing step 3 of GeneFace-1.1.0 while running the command "CUDA_VISIBLE_DEVICES=1 data_gen/nerf/process_data.sh $VIDEO_ID" given under "process_target_person_video.md" file to process May.mp4 video. It executes till step 8 and for step 9 which calculates 3DMM, it crashes throwing several errors.