instant-high / wav2lip-onnx-HQ

Full version of wav2lip-onnx including face alignment and face enhancement and more...
63 stars 11 forks source link

i can't run it ! #5

Open MohamedHamdy98 opened 2 months ago

MohamedHamdy98 commented 2 months ago

i try to run it after setup and i got this error : Reading video frames... Select a ROI and then press SPACE or ENTER button! Cancel the selection process by pressing c button! qt.qpa.xcb: could not connect to display qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/opt/conda/lib/python3.10/site-packages/cv2/qt/plugins" even though it was found. This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: xcb.

this my run : !python /kaggle/working/wav2lip-onnx-HQ/inference_onnxModel.py \ --checkpoint_path /kaggle/working/wav2lip-onnx-HQ/checkpoints/wav2lip_onnex_models.pth \ --face /kaggle/input/videos-deepfake/sa.mp4 \ --audio /kaggle/input/videos-deepfake/vae.wav \ --outfile /kaggle/working/output.mp4

instant-high commented 2 months ago

You have to download all ziped .onnx models and place them in the correct folders. Looks like you are trying to load an torch model .pth https://drive.google.com/drive/folders/1BGl9bmMtlGEMx_wwKufJrZChFyqjnlsQ?usp=sharing

MohamedHamdy98 commented 2 months ago

You have to download all ziped .onnx models and place them in the correct folders. Looks like you are trying to load an torch model .pth https://drive.google.com/drive/folders/1BGl9bmMtlGEMx_wwKufJrZChFyqjnlsQ?usp=sharing

i actually did it! , i downloaded and unzip all of them this my code to did this , and still got the same error: import os import gdown import zipfile

List of Google Drive links

drive_links = [ "https://drive.google.com/file/d/1_l4QC2RJ9nXapSQRD61-Q4KbSApc53HM/view?usp=drive_link", "https://drive.google.com/file/d/1av7_NtpNxcgZxnLLmE9Ymu70bvZWmZrg/view?usp=drive_link", "https://drive.google.com/file/d/1jXhj6GEKW_ypJjhf_zHlAR_hQJnm_uap/view?usp=drive_link", "https://drive.google.com/file/d/18OFqPuGYePXwvEvfvu99GFaLlyxrHnE-/view?usp=drive_link", "https://drive.google.com/file/d/1UIwCLiS5bb7nEfqasnXboCOAVKPzI5oP/view?usp=drive_link" ]

Directory to save the files

output_dir = "/kaggle/working/wav2lip-onnx-HQ/checkpoints"

Create the directory if it doesn't exist

os.makedirs(output_dir, exist_ok=True)

for link in drive_links:

Extract the file ID from the link

file_id = link.split('/d/')[1].split('/view')[0]
download_url = f"https://drive.google.com/uc?id={file_id}"

# Define the output file path
output_path = os.path.join(output_dir, f"{file_id}.zip")

# Download the file
gdown.download(download_url, output_path, quiet=False)

# Unzip the file
with zipfile.ZipFile(output_path, 'r') as zip_ref:
    zip_ref.extractall(output_dir)

# Optionally, remove the zip file after extraction
os.remove(output_path)

Verify the contents

!ls {output_dir}

if you have example to run it can you pls share it! thanks 😀

MohamedHamdy98 commented 2 months ago

this my files in my checkPoints file: image

instant-high commented 2 months ago

Which OS are you using. Looks quite complicated what you are doing. When you have downloaded the zip files, on Windows use winzip or 7zip and extract the *.onnx files to eg. 'wav2lip.onnx' and 'wav2lip_ gan.onnx' to the folder checkpoints. Insightface model should already be in the correct folder. The faceenhancers go to eg. 'enhancers/codeformer/codeformer.onnx'

This is a full onnx based repo, so there is no wav2lip_onnex_models.pth. Same for all other used models.

instant-high commented 2 months ago

..../checkpoints/wav2lip.onnx ..../checkpoints/wav2lip_gan.onnx ..../enhancers/codeformer/codeformer.onnx

and so on

instant-high commented 2 months ago

I recommend using anaconda.

eg. to run wav2lip_onnx using anaconda: (minimum commandline)

call C:\your_path_to\Anaconda3\Scripts\activate.bat call conda activate ENV_NAME cd c:\your_path_to\wav2lip-ONNX-HQ python inference_onnxModel.py --checkpoint_path "checkpoints/wav2lip.onnx" --face "full_path_to_image_or_video.mp4" --audio "full_path_to_audio.wav" --outfile "full_path_to_output.mp4"

MohamedHamdy98 commented 2 months ago

i did all of this but still the same error , i run it on google colab or Kaggle not anaconda because (!pip install insightface==0.7.3) can't install in anaconda

instant-high commented 2 months ago

This repository is for local installation using anaconda. I do not support colab or kaggle.

If you have problems with pip install insightface==0.7.3 try download from here: https://github.com/Gourieff/Assets/blob/main/Insightface/insightface-0.7.3-cp311-cp311-win_amd64.whl copy to your wav2lip-hq directory and run: pip install insightface-0.7.3-cp311-cp311-win_amd64.whl

(I also have to do it that way)

Or try pip install insightface==0.2.1 Should also work I think