Errors when trying to animate and other noob questions

[AlienX971]

Hi guys,

So, I'm trying to animate a video with the vox-adv-256 model with the provided command CUDA_VISIBLE_DEVICES=0 python run.py --config config/dataset_name.yaml --mode animate --checkpoint path/to/checkpoint but I encounter various problems.

first is the CUDA_VISIBLE_DEVICES=0 part. Although I have an NVidia GPU which is a CUDA device, for whatever reason I get the error 'CUDA_VISIBLE_DEVICES' is not recognized as an internal or external command, operable program or batch file. I am using Anaconda3 for the Python environment, do I need some additional dependencies, other than the ones provided here, in order for it to recognize this command?

Anyway, I decide to run the script without this part and I keep running into more problems. The vox-adv-256 dataset doesn't have its own provided .csv file, from which to look for the files. So I made my own in the required path, and after some formatting issues, I finally got it somewhat working, but then I get another huge problem. The script seems to run, it creates the "Animation" and "png" folders it starts to animate, but then stops and doesn't produce any results. At this point I am completely stumped and I wonder if anyone can tell me how to get this to work properly?

I ran the animation demo and that worked fine, but the results I got were very unrealistic. So I thought that's because it's just a demo and I should just try the real scripts, but I can't seem to get them working. I would like to know what I am doing wrong and how to properly set up everything and do various things with these scripts? They can be used to achieve some really impressive things. Any help would be greatly appreciated. :)

[AliaksandrSiarohin]

No demo, produce same results as animate. Most like your video is not cropped. Crop the video manually or use crop_video script.

[AlienX971]

OK, so I tried to crop the video with the script and the face alignment tool, but ran into more issues. First it wanted an even newer version of PyTorch for whatever reason. I installed it, tried running the script and I get this error:

python crop-video.py --inp videos/TrumpFull.mp4 Traceback (most recent call last): File "crop-video.py", line 1, in import face_alignment File "D:\Anaconda3\envs\Animation\lib\site-packages\face_alignment-1.3.3-py3.7.egg\face_alignment__init.py", line 7, in File "D:\Anaconda3\envs\Animation\lib\site-packages\face_alignment-1.3.3-py3.7.egg\face_alignment\api.py", line 1, in File "D:\Anaconda3\envs\Animation\lib\site-packages\torch\init__.py", line 136, in from torch._C import * ImportError: numpy.core.multiarray failed to import

I have no idea what I'm doing wrong exactly.

[AlienX971]

I just noticed you said I could crop it manually, which I initially did when running the demo. I'm not sure if I did it incorrectly or if the tool does something differently. Here are gifs of the driving video and the result, in case anyone can see if anything is wrong with the cropping I did.

You can see I picked an image where the head position is relatively similar, but you can see the results are not that good. I really don't know what exactly I did wrong?

[AliaksandrSiarohin]

Looks good enough to me. I don't think It produce much better results.

[AlienX971]

You sure? The demonstration animations on the page and some other results people have achieved look a lot more realistic and definitely not as warped. Not sure what they did differently. Would using .png files in sequence instead of a single video produce better results? How do you set them as an input in this case?

[AliaksandrSiarohin]

In the page I used frontal faces, which it simpler. No it won't help.