search

lelechen63 / Talking-head-Generation-with-Rhythmic-Head-Motion

Other
197 stars 28 forks source link

Error in single_video_preprocess.py #4

Open ssinha89 opened 3 years ago

ssinha89 commented 3 years ago

Thanks for releasing the data preprocessing code. However, I am unable to run the single_video_preprocess.py file due to the following error: ModuleNotFoundError: No module named 'face_tracker'

Also it would be helpful if the details on generating the 3d animation video aligned_ani.mp4 was provided, as it is mentioned that the animation video is needed when the --warp_ani flag is set.

lelechen63 commented 3 years ago

Sorry I have not test the code. The face_tracker is another file. I will update this later today. I am preparing the 3D animation part. That part is large so I am figuring out how to upload to github.

lelechen63 commented 3 years ago

Thanks for releasing the data preprocessing code. However, I am unable to run the single_video_preprocess.py file due to the following error: ModuleNotFoundError: No module named 'face_tracker'

Also it would be helpful if the details on generating the 3d animation video aligned_ani.mp4 was provided, as it is mentioned that the animation video is needed when the --warp_ani flag is set.

Please visit https://github.com/1adrianb/face-alignment to install face_alignment package.

Adorablepet commented 3 years ago

@ssinha89 In sigle_video_preprocess.py

command = 'ffmpeg -framerate 25 -i ./temp%05d'%config.batch_id + '/%05d.png -vcodec libx264 -vf format=yuv420p -y ' +  cropped_video_path
          ^
SyntaxError: invalid syntax

how to deal with this error?

Adorablepet commented 3 years ago

single_video_preprocess.py TAB string exists, replace it with four spaces. It's ok. modify 70 line, as follows:

try:
    try:
         _crop_video(original_video_path, config.batch_id,  1)
    except:
         print('some error when crop images.')
except:
     pass
ssinha89 commented 3 years ago

Sorry I have not test the code. The face_tracker is another file. I will update this later today. I am preparing the 3D animation part. That part is large so I am figuring out how to upload to github.

@lelechen63 Thanks, looking forward to the details on preparing the input 3d animation video. I am wondering how to evaluate the entire pipeline model on audio input? Currently only the model for evaluating head pose from audio input is provided, could you also share demo model and code for deriving facial expressions on landmarks from input audio? The demo code for texture generation takes landmarks as input, however it is not clear from the existing code how to generate the landmarks from input audio.

dipam7 commented 3 years ago

The face_tracker is another file. I will update this later today.

Hey, any updates on this? Really looking forward to trying it

lelechen63 commented 3 years ago

The face_tracker is another file. I will update this later today.

Hey, any updates on this? Really looking forward to trying it

Check the readme in data folder.

dipam7 commented 3 years ago

Check the readme in data folder.

I did, I don't see the face_tracker file or any workarounds. I'm running the following command.

python single_video_preprocess.py --extract_landmark --video_path=../demo/00181_aligned.mp4

And here's the list of packages I've installed. Am I doing something wrong?

$pip list
Package         Version  
--------------- ---------
certifi         2020.6.20
cycler          0.10.0   
decorator       4.4.2    
dlib            19.21.0  
dominate        2.5.2    
face-alignment  1.1.0    
imageio         2.9.0    
kiwisolver      1.2.0    
matplotlib      3.3.1    
networkx        2.5      
numpy           1.19.1   
opencv-python   4.4.0.42 
Pillow          7.2.0    
pip             19.0.3   
pyparsing       2.4.7    
python-dateutil 2.8.1    
pytorch-msssim  0.2.1    
PyWavelets      1.1.1    
scikit-image    0.17.2   
scipy           1.5.2    
setuptools      40.8.0   
six             1.15.0   
tifffile        2020.8.25
torch           1.4.0    
torchvision     0.5.0    
tqdm            4.48.2
lelechen63 commented 3 years ago

Updated.

From: Dipam Vasani notifications@github.com Sent: Saturday, August 29, 2020 3:34 PM To: lelechen63/Talking-head-Generation-with-Rhythmic-Head-Motion Talking-head-Generation-with-Rhythmic-Head-Motion@noreply.github.com Cc: Chen, Lele l.chen@rochester.edu; Mention mention@noreply.github.com Subject: [EXT] Re: [lelechen63/Talking-head-Generation-with-Rhythmic-Head-Motion] Error in single_video_preprocess.py (#4)

Check the readme in data folder.

I did, I don't see the face_tracker file or any workarounds. I'm running the following command.

python single_video_preprocess.py --extract_landmark --video_path=../demo/00181_aligned.mp4

And here's the list of packages I've installed. Am I doing something wrong?

$pip list Package Version

certifi 2020.6.20 cycler 0.10.0 decorator 4.4.2 dlib 19.21.0 dominate 2.5.2 face-alignment 1.1.0 imageio 2.9.0 kiwisolver 1.2.0 matplotlib 3.3.1 networkx 2.5 numpy 1.19.1 opencv-python 4.4.0.42 Pillow 7.2.0 pip 19.0.3 pyparsing 2.4.7 python-dateutil 2.8.1 pytorch-msssim 0.2.1 PyWavelets 1.1.1 scikit-image 0.17.2 scipy 1.5.2 setuptools 40.8.0 six 1.15.0 tifffile 2020.8.25 torch 1.4.0 torchvision 0.5.0 tqdm 4.48.2

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_lelechen63_Talking-2Dhead-2DGeneration-2Dwith-2DRhythmic-2DHead-2DMotion_issues_4-23issuecomment-2D683333279&d=DwMCaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=qRrg4VTSFIYKRMz88LmmxufThi8eOutCmC64X_B4jVc&m=hXfJF45WmBPeioP1nVK6ZCHVdV7IhM3-HcEF7YV7wdQ&s=wlSEMw0P6VwJhOR7KXuC2jni9DWP7bd_Ztd_HCUvi7o&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AF4TJXDEGXMII45W7UEBHPDSDFJ3HANCNFSM4QK6SUDQ&d=DwMCaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=qRrg4VTSFIYKRMz88LmmxufThi8eOutCmC64X_B4jVc&m=hXfJF45WmBPeioP1nVK6ZCHVdV7IhM3-HcEF7YV7wdQ&s=E1cp4-4d7BwPi9tQySNPnXBzatA77TgxBJqTSM-rEAA&e=.

dipam7 commented 3 years ago

Thanks, getting another error now

ModuleNotFoundError: No module named 'utils'

and if I do

pip install utils

it gives me

ModuleNotFoundError: No module named 'utils.util'

Can you do a quick pip freeze > requirements.txt and upload the requirements.txt? That would be really helpful.

lelechen63 commented 3 years ago

Hi there,

The code has not been tested yet. I will test it later. I will aslo upload utils file later.

Thanks.

Best, Lele

From: Dipam Vasani notifications@github.com Sent: Saturday, August 29, 2020 3:44 PM To: lelechen63/Talking-head-Generation-with-Rhythmic-Head-Motion Talking-head-Generation-with-Rhythmic-Head-Motion@noreply.github.com Cc: Chen, Lele l.chen@rochester.edu; Mention mention@noreply.github.com Subject: [EXT] Re: [lelechen63/Talking-head-Generation-with-Rhythmic-Head-Motion] Error in single_video_preprocess.py (#4)

Thanks, getting another error now

ModuleNotFoundError: No module named 'utils'

and if I do

pip install utils

it gives me

ModuleNotFoundError: No module named 'utils.util'

Can you do a quick pip freeze > requirements.txt and upload the requirements.txt? That would be really helpful.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_lelechen63_Talking-2Dhead-2DGeneration-2Dwith-2DRhythmic-2DHead-2DMotion_issues_4-23issuecomment-2D683334137&d=DwMCaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=qRrg4VTSFIYKRMz88LmmxufThi8eOutCmC64X_B4jVc&m=WlZoInBWE_dSVPdF3zCveGpS72fvrsw357MUD-v_VHE&s=3h3kW9ZSyYFP06FvJK-C3MEhQBokpRoLUR2abpGuGFE&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AF4TJXE7WBE7PMNXLQ3CKE3SDFLAHANCNFSM4QK6SUDQ&d=DwMCaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=qRrg4VTSFIYKRMz88LmmxufThi8eOutCmC64X_B4jVc&m=WlZoInBWE_dSVPdF3zCveGpS72fvrsw357MUD-v_VHE&s=cRqGltBaz6ouJod_quEwWddBr7dcAjEQHFPDh_W_Y9M&e=.

ak9250 commented 3 years ago

@lelechen63 would be helpful to have a google colab for the demo, that way there are less environment issues

ankushbikkasani commented 3 years ago

@lelechen63 getting the same error as @dipam7. Would appreciate utils file. Thanks

nbaker1997 commented 3 years ago

@lelechen63 Having same issue as @dipam7 as well. Utils file would be great, cheers!

tangyibo123 commented 3 years ago

you can rename 'util' folder to 'utils'

tangyibo123 commented 3 years ago

Thanks, getting another error now

ModuleNotFoundError: No module named 'utils'

and if I do

pip install utils

it gives me

ModuleNotFoundError: No module named 'utils.util'

Can you do a quick pip freeze > requirements.txt and upload the requirements.txt? That would be really helpful.

you can rename 'util' folder to 'utils'

lelechen63 commented 3 years ago

It should be good to go. I already tested all the functions in single_video_preprocess.py.

ssinha89 commented 3 years ago

@lelechen63 Thanks for updating the 3d animation data preparation code. It would also be great if you could share the details of the expression generation from audio as mentioned here.

@lelechen63 Thanks, looking forward to the details on preparing the input 3d animation video. I am wondering how to evaluate the entire pipeline model on audio input? Currently only the model for evaluating head pose from audio input is provided, could you also share demo model and code for deriving facial expressions on landmarks from input audio? The demo code for texture generation takes landmarks as input, however it is not clear from the existing code how to generate the landmarks from input audio.

ak9250 commented 3 years ago

@ssinha89 were you able to get this working, do you have a google colab notebook for the demo?

Adorablepet commented 3 years ago

@lelechen63 FileNotFoundError: [Errno 2] No such file or directory: '../basics/standard.npy' in single_video_preprocess.py

lelechen63 commented 3 years ago

@lelechen63 FileNotFoundError: [Errno 2] No such file or directory: '../basics/standard.npy' in single_video_preprocess.py

uploaded

Adorablepet commented 3 years ago

@lelechen63 ValueError: The passed save_path is not a valid checkpoint: ./Data/net-data/256_256_resfcn256_weight, I have downloaded the PRN trained model at GoogleDrive, and put it into Data/net-data.

lelechen63 commented 3 years ago

I will fix it tomorrow.

Sent via the Samsung Galaxy S10, an AT&T 5G Evolution capable smartphone Get Outlook for Androidhttps://aka.ms/ghei36

From: Pearson Zhao notifications@github.com Sent: Monday, August 31, 2020 12:31:11 AM To: lelechen63/Talking-head-Generation-with-Rhythmic-Head-Motion Talking-head-Generation-with-Rhythmic-Head-Motion@noreply.github.com Cc: Chen, Lele l.chen@rochester.edu; Mention mention@noreply.github.com Subject: [EXT] Re: [lelechen63/Talking-head-Generation-with-Rhythmic-Head-Motion] Error in single_video_preprocess.py (#4)

@lelechen63https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_lelechen63&d=DwMCaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=qRrg4VTSFIYKRMz88LmmxufThi8eOutCmC64X_B4jVc&m=l2xUJO22u6Yj8SItiqv8KAqO9xBTGnb7hTuQssNq0Sw&s=A966iVW3n11djyVAj8fsVKLcKCRdiQu79tE0POwD2zw&e= ValueError: The passed save_path is not a valid checkpoint: ./Data/net-data/256_256_resfcn256_weight, I have downloaded the PRN trained model at GoogleDrive, and put it into Data/net-data.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_lelechen63_Talking-2Dhead-2DGeneration-2Dwith-2DRhythmic-2DHead-2DMotion_issues_4-23issuecomment-2D683542845&d=DwMCaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=qRrg4VTSFIYKRMz88LmmxufThi8eOutCmC64X_B4jVc&m=l2xUJO22u6Yj8SItiqv8KAqO9xBTGnb7hTuQssNq0Sw&s=BP8pKth5Ju3EBHgfiQMfzxY2R6sstnJxod6gnM8NN1s&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AF4TJXECZCAODHRYA4IAICTSDMRQ7ANCNFSM4QK6SUDQ&d=DwMCaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=qRrg4VTSFIYKRMz88LmmxufThi8eOutCmC64X_B4jVc&m=l2xUJO22u6Yj8SItiqv8KAqO9xBTGnb7hTuQssNq0Sw&s=7dcJdOaCQS8Vdr-ZcwKiRkm0r7Axr8OmmftowUhVUXE&e=.

Adorablepet commented 3 years ago

I have solved from https://github.com/clks-wzz/PRNet-Depth-Generation/issues/3, In Data/net-data add 256_256_resfcn256_weight.index file. I don’t know if this is the case.

tangyibo123 commented 3 years ago

@lelechen63
File "find_camera.py", line 2, in import soft_renderer as sr ModuleNotFoundError: No module named 'soft_renderer'

Adorablepet commented 3 years ago

@lelechen63 if direction.ndimension() == 1: AttributeError: 'bool' object has no attribute 'ndimension'. I have downloaded SoftRas project, and python setup.py install in conda envs.

python 3.6.10
pytorch 1.1.0
cuda 10.0.130
cudnn 7.6.5

I tested my video, and what caused this error? Thanks.

Adorablepet commented 3 years ago

@lelechen63 TypeError: __init__() got an unexpected keyword argument 'camera_up', camera_up parametrer unnecessary?

ssinha89 commented 3 years ago

@lelechen63 TypeError: __init__() got an unexpected keyword argument 'camera_up', camera_up parametrer unnecessary?

I got the same error. On removing that argument, I got the error

File "/usr/local/lib/python3.7/dist-packages/soft_renderer-1.0.0-py3.7-linux-x86_64.egg/soft_renderer/transform.py", line 65, in forward vertices = srf.look(vertices, self._eye, self.camera_direction) File "/usr/local/lib/python3.7/dist-packages/soft_renderer-1.0.0-py3.7-linux-x86_64.egg/soft_renderer/functional/look.py", line 31, in look if direction.ndimension() == 1: AttributeError: 'bool' object has no attribute 'ndimension'

lelechen63 commented 3 years ago

@lelechen63 TypeError: __init__() got an unexpected keyword argument 'camera_up', camera_up parametrer unnecessary?

I got the same error. On removing that argument, I got the error

File "/usr/local/lib/python3.7/dist-packages/soft_renderer-1.0.0-py3.7-linux-x86_64.egg/soft_renderer/transform.py", line 65, in forward vertices = srf.look(vertices, self._eye, self.camera_direction) File "/usr/local/lib/python3.7/dist-packages/soft_renderer-1.0.0-py3.7-linux-x86_64.egg/soft_renderer/functional/look.py", line 31, in look if direction.ndimension() == 1: AttributeError: 'bool' object has no attribute 'ndimension'

You need to use my softrs to install it. Instead of the official one. You need to cd to my softrs folder and run the installation.

tangyibo123 commented 3 years ago

@lelechen63 Tanks，are all the codes updated? I can't import soft_renderer

lelechen63 commented 3 years ago

@lelechen63 Tanks，are all the codes updated? I can't import soft_renderer

Yes. All the functions in single_video_process.py have been tested. You need to go to softrs folder to install it.

ssinha89 commented 3 years ago

@lelechen63 TypeError: __init__() got an unexpected keyword argument 'camera_up', camera_up parametrer unnecessary?

I got the same error. On removing that argument, I got the error File "/usr/local/lib/python3.7/dist-packages/soft_renderer-1.0.0-py3.7-linux-x86_64.egg/soft_renderer/transform.py", line 65, in forward vertices = srf.look(vertices, self._eye, self.camera_direction) File "/usr/local/lib/python3.7/dist-packages/soft_renderer-1.0.0-py3.7-linux-x86_64.egg/soft_renderer/functional/look.py", line 31, in look if direction.ndimension() == 1: AttributeError: 'bool' object has no attribute 'ndimension'

You need to use my softrs to install it. Instead of the official one. You need to cd to my softrs folder and run the installation.

The SoftRas folder inside data/face-tools is empty. How to run your version of softras?

lelechen63 commented 3 years ago

@lelechen63 TypeError: __init__() got an unexpected keyword argument 'camera_up', camera_up parametrer unnecessary?

I got the same error. On removing that argument, I got the error File "/usr/local/lib/python3.7/dist-packages/soft_renderer-1.0.0-py3.7-linux-x86_64.egg/soft_renderer/transform.py", line 65, in forward vertices = srf.look(vertices, self._eye, self.camera_direction) File "/usr/local/lib/python3.7/dist-packages/soft_renderer-1.0.0-py3.7-linux-x86_64.egg/soft_renderer/functional/look.py", line 31, in look if direction.ndimension() == 1: AttributeError: 'bool' object has no attribute 'ndimension'

You need to use my softrs to install it. Instead of the official one. You need to cd to my softrs folder and run the installation.

The SoftRas folder inside data/face-tools is empty. How to run your version of softras?

Please git clone the softras folder from https://github.com/goddice/SoftRas

neeek2303 commented 1 year ago

@lelechen63 I've got an error when try to import your version: ModuleNotFoundError: No module named 'soft_renderer.cuda.load_textures'