search

OpenTalker / StyleHEAT

[ECCV 2022] StyleHEAT: A framework for high-resolution editable talking face generation
MIT License
620 stars 77 forks source link

How to test the Audio Driven？ #9

Open c1a1o1 opened 2 years ago

c1a1o1 commented 2 years ago

How to test the Audio Driven？

c1a1o1 commented 2 years ago

D:\anaconda\python.exe "D:/work/vgan/StyleHEAT-main/inference - test.py" Cross-id testing Enable audio driven. Load pre-trained e4e Encoder from checkpoints/Encoder_e4e.pth done. Load pre-trained hfgi encoder from checkpoints/hfgi.pth done. Load pre-trained StyleGAN2 from checkpoints/StyleGAN_e4e.pth done. Stage: inference Load pre-trained StyleHEAT [net_G_ema] from checkpoints/StyleHEAT_visual.pt done ----------------- Options --------------- add_image: True
bfm_folder: D:/work/vgan/StyleHEAT-main/checkpoints/BFM [default: BFM] bfm_model: BFM_model_front.mat
camera_d: 10.0
center: 112.0
checkpoints_dir: D:/work/vgan/StyleHEAT-main/checkpoints [default: ./checkpoints] dataset_mode: None
ddp_port: 12355
display_per_batch: True
epoch: 20 [default: latest] eval_batch_nums: inf
focal: 1015.0
gpu_ids: 0
img_folder: temp [default: examples] init_path: checkpoints/init_model/resnet50-0676ba61.pth isTrain: False [default: None] model: facerecon
name: model_name [default: face_recon] net_recon: resnet50
phase: test
suffix:
use_ddp: False [default: True] use_last_fc: False
verbose: False
vis_batch_nums: 1
world_size: 1
z_far: 15.0
z_near: 5.0
----------------- End ------------------- model [FaceReconModel] was created loading the model from D:/work/vgan/StyleHEAT-main/checkpoints\model_name\epoch_20.pth 0%| | 0/1 [00:00<?, ?it/s]D:\anaconda\lib\site-packages\torch\nn\functional.py:4194: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details. "Default grid_sample and affine_grid behavior has changed " Traceback (most recent call last): File "D:/work/vgan/StyleHEAT-main/inference - test.py", line 224, in main() File "D:/work/vgan/StyleHEAT-main/inference - test.py", line 212, in main reenactment(generator, data) File "D:/work/vgan/StyleHEAT-main/inference - test.py", line 59, in reenactment output = generator.forward(source_image, target_3dmm, inv_data=inv_data, imsize=1024) File "D:\work\vgan\StyleHEAT-main\models\styleheat\styleheat.py", line 101, in forward flow = self.audio_warper(video_warp_img_256, driven_audio)['flow_field'] # Input: 256256 File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl return forward_call(input, kwargs) File "D:\work\vgan\StyleHEAT-main\models\styleheat\warper.py", line 72, in forward descriptor = self.audio_encoder(driving_source) File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl return forward_call(*input, *kwargs) File "D:\work\vgan\StyleHEAT-main\models\styleheat\warper.py", line 99, in forward out = self.first(input_3dmm) File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl return forward_call(input, kwargs) File "D:\anaconda\lib\site-packages\torch\nn\modules\container.py", line 141, in forward input = module(input) File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl return forward_call(*input, **kwargs) File "D:\anaconda\lib\site-packages\torch\nn\modules\conv.py", line 302, in forward return self._conv_forward(input, self.weight, self.bias) File "D:\anaconda\lib\site-packages\torch\nn\modules\conv.py", line 299, in _conv_forward self.padding, self.dilation, self.groups) TypeError: conv1d() received an invalid combination of arguments - got (NoneType, Parameter, Parameter, tuple, tuple, tuple, int), but expected one of:

Process finished with exit code 1

simasima121 commented 1 year ago

D:\anaconda\python.exe "D:/work/vgan/StyleHEAT-main/inference - test.py" Cross-id testing Enable audio driven. Load pre-trained e4e Encoder from checkpoints/Encoder_e4e.pth done. Load pre-trained hfgi encoder from checkpoints/hfgi.pth done. Load pre-trained StyleGAN2 from checkpoints/StyleGAN_e4e.pth done. Stage: inference Load pre-trained StyleHEAT [net_G_ema] from checkpoints/StyleHEAT_visual.pt done ----------------- Options --------------- add_image: True bfm_folder: D:/work/vgan/StyleHEAT-main/checkpoints/BFM [default: BFM] bfm_model: BFM_model_front.mat camera_d: 10.0 center: 112.0 checkpoints_dir: D:/work/vgan/StyleHEAT-main/checkpoints [default: ./checkpoints] dataset_mode: None ddp_port: 12355 display_per_batch: True epoch: 20 [default: latest] eval_batch_nums: inf focal: 1015.0 gpu_ids: 0 img_folder: temp [default: examples] init_path: checkpoints/init_model/resnet50-0676ba61.pth isTrain: False [default: None] model: facerecon name: model_name [default: face_recon] net_recon: resnet50 phase: test suffix: use_ddp: False [default: True] use_last_fc: False verbose: False vis_batch_nums: 1 world_size: 1 z_far: 15.0 z_near: 5.0 ----------------- End ------------------- model [FaceReconModel] was created loading the model from D:/work/vgan/StyleHEAT-main/checkpoints\model_name\epoch_20.pth 0%| | 0/1 [00:00<?, ?it/s]D:\anaconda\lib\site-packages\torch\nn\functional.py:4194: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details. "Default grid_sample and affine_grid behavior has changed " Traceback (most recent call last): File "D:/work/vgan/StyleHEAT-main/inference - test.py", line 224, in main() File "D:/work/vgan/StyleHEAT-main/inference - test.py", line 212, in main reenactment(generator, data) File "D:/work/vgan/StyleHEAT-main/inference - test.py", line 59, in reenactment output = generator.forward(source_image, target_3dmm, inv_data=inv_data, imsize=1024) File "D:\work\vgan\StyleHEAT-main\models\styleheat\styleheat.py", line 101, in forward flow = self.audio_warper(video_warp_img_256, driven_audio)['flow_field'] # Input: 256256 File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl return forward_call(input, kwargs) File "D:\work\vgan\StyleHEAT-main\models\styleheat\warper.py", line 72, in forward descriptor = self.audio_encoder(driving_source) File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl return forward_call(*input, *kwargs) File "D:\work\vgan\StyleHEAT-main\models\styleheat\warper.py", line 99, in forward out = self.first(input_3dmm) File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl return forward_call(input, kwargs) File "D:\anaconda\lib\site-packages\torch\nn\modules\container.py", line 141, in forward input = module(input) File "D:\anaconda\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl return forward_call(*input, **kwargs) File "D:\anaconda\lib\site-packages\torch\nn\modules\conv.py", line 302, in forward return self._conv_forward(input, self.weight, self.bias) File "D:\anaconda\lib\site-packages\torch\nn\modules\conv.py", line 299, in _conv_forward self.padding, self.dilation, self.groups) TypeError: conv1d() received an invalid combination of arguments - got (NoneType, Parameter, Parameter, tuple, tuple, tuple, int), but expected one of:

  • (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, tuple of ints padding, tuple of ints dilation, int groups) didn't match because some of the arguments have invalid types: (!NoneType!, !Parameter!, !Parameter!, !tuple!, !tuple!, !tuple!, int)
  • (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, str padding, tuple of ints dilation, int groups) didn't match because some of the arguments have invalid types: (!NoneType!, !Parameter!, !Parameter!, !tuple!, !tuple!, !tuple!, int)

0%| | 0/1 [00:14<?, ?it/s]

Process finished with exit code 1

Do you have the audio driven model ?