No checkpoint at 2000

[jasonlee328]

I was able to run the inference but there's still no checkpoint at 2000. The output result is a video at iteration 00000 which has no mouth movement.

Perceptual loss: Mode: vgg19 Perceptual loss: Mode: vgg19 Perceptual loss: Mode: vgg19 Perceptual loss: Mode: vgg19 Loading ResNet ArcFace loading id loss module: Loading ResNet ArcFace loading id loss module: Loss perceptual_inverse_lr Weight 1.0 Loss perceptual_inverse_sr Weight 1.0 Loss perceptual_refine_lr Weight 1.0 Loss perceptual_refine_sr Weight 1.0 Loss monotonic Weight 1.0 Loss TV Weight 1.0 Loss pixel Weight 1 Loss a_norm Weight 0.0 Loss a_mutual Weight 0.0 Loss local Weight 10.0 Loss local_s Weight 10.0 Loss id Weight 1.0 Loss id_s Weight 1.0 We train Generator No checkpoint found at iteration 2000. 0%| | 0/19 [00:00<?, ?it/s] 0%| | 0/19 [00:07<?, ?it/s]

[kunyao2015]

Same error; But I have other error. file models/controller.py : Q, R = torch.linalg.qr(weight) torch._C._LinAlgError: cusolver error: CUSOLVER_STATUS_EXECUTION_FAILED, when calling 'cusolverDnXgeqef' My pytorch version = 1.12.1 ; CUDA Version=11.0; Nvidia TiTAN V card; Drivier Version=450.80.02

And what's your environment？

[theEricMa]

@jaslee20 Please check the latest update for the uploaded model. Thank you!

[theEricMa]

@kunyao2015 Our model is trained and evaluated on A100, we will further examine our model on other GPU platforms. I suppose your cuda driver is installed uncorrectly.

[jasonlee328]

@theEricMa Thank you! I was able to run the model successfully. In the next step I want to experiment with my own data and images. When do you think you will release the code for processing data, or is it possible to use PIRender's processing for now. Great work and thanks.

[87003697]

@jaslee20 Yeah we plan to release the preprocessing script recently.

[kunyao2015]

@theEricMa Thanks for your replay! when the CUDA Version=11.7 or 12.0 ， the model can run successfully. I have another question, how can I get the 'condition' params from one video or image?

[theEricMa]

@kunyao2015 Our method uses ADNeRF to get the camera pose. You can resize the intrinsic (3 $*$ 3) and extrinsic (4 $*$ 4) to be a 25-dimensional condition. If you want to reproduce to the result before the next release, pelase feel free to contact me if you meet problems.