ashawkey / RAD-NeRF

Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decomposition
MIT License
862 stars 151 forks source link

Why are there no "obama.json" and "obama.pth" files after the training is completed? These two files are required for inference after training, but they were not generated even though the training was successful. #55

Open shiyueGG opened 1 year ago

shiyueGG commented 1 year ago

Why are there no "obama.json" and "obama.pth" files after the training is completed? These two files are required for inference after training, but they were not generated even though the training was successful.

./data/ ├──.mp4 # original video ├──ori_imgs # original images from video │ ├──0.jpg │ ├──0.lms # 2D landmarks │ ├──... ├──gt_imgs # ground truth images (static background) │ ├──0.jpg │ ├──... ├──parsing # semantic segmentation │ ├──0.png │ ├──... ├──torso_imgs # inpainted torso images │ ├──0.png │ ├──... ├──aud.wav # original audio ├──aud_eo.npy # audio features (wav2vec) ├──aud.npy # audio features (deepspeech) ├──bc.jpg # default background ├──track_params.pt # raw head tracking results ├──transforms_train.json # head poses (train split) ├──transforms_val.json # head poses (test split)

iboyles commented 12 months ago

I agree I have just finishing training but it won't inference correctly. I need the colab to be updated for inferencing with a custom trained model and more documentation.

hg111 commented 10 months ago

@shiyueGG @iboyles did you ever figure out why the obama.json and obama.pth aren't generated?

iboyles commented 8 months ago

you probably ran out of compute time during proccessing, I think I ran this on google colab also may not just work in jupyter environment.

iboyles commented 8 months ago

The first processing steps are compute heavy and take a few hours.