ali-vilab / UniAnimate

Code for Paper "UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation".
https://unianimate.github.io/
524 stars 29 forks source link

data/saved_pose/musk-42 read video frame failed with error: OpenCV(4.5.3) /tmp/pip-req-build-3umofm98/opencv/modules/imgproc/src/color.cpp:182: error: #18

Open shuodehaoa opened 1 week ago

shuodehaoa commented 1 week ago

[2024-06-18 14:49:24,912] INFO: There are 1 videos. with 1 times [2024-06-18 14:49:24,912] INFO: [0]/[1] Begin to sample data/images/musk.jpg, pose sequence from data/saved_pose/musk-42 init seed 11 ... [2024-06-18 14:50:02,486] INFO: data/saved_pose/musk-42 read video frame failed with error: OpenCV(4.5.3) /tmp/pip-req-build-3umofm98/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

[2024-06-18 14:50:38,144] INFO: data/saved_pose/musk-42 read video frame failed with error: OpenCV(4.5.3) /tmp/pip-req-build-3umofm98/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

[2024-06-18 14:51:13,798] INFO: data/saved_pose/musk-42 read video frame failed with error: OpenCV(4.5.3) /tmp/pip-req-build-3umofm98/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

[2024-06-18 14:51:49,441] INFO: data/saved_pose/musk-42 read video frame failed with error: OpenCV(4.5.3) /tmp/pip-req-build-3umofm98/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

shuodehaoa commented 1 week ago

In stage of Pose alignment, when i run inference.py in my uplated video, it show above error.

wangxiang1230 commented 1 week ago

In stage of Pose alignment, when i run inference.py in my uplated video, it show above error.

Hi,in the stage of pose alignment, you need to run the command like “python run_align_pose.py --ref_name data/images/WOMEN-Blouses_Shirts-id_00004955-01_4_full.jpg --source_video_paths data/videos/source_video.mp4 --saved_pose_dir data/saved_pose/WOMEN-Blouses_Shirts-id_00004955-01_4_full”.

shuodehaoa commented 1 week ago

In stage of Pose alignment, when i run inference.py in my uplated video, it show above error.

Hi,in the stage of pose alignment, you need to run the command like “python run_align_pose.py --ref_name data/images/WOMEN-Blouses_Shirts-id_00004955-01_4_full.jpg --source_video_paths data/videos/source_video.mp4 --saved_pose_dir data/saved_pose/WOMEN-Blouses_Shirts-id_00004955-01_4_full”.

I'am sorry, I've just described it incorrectly. I've done the pose alignment with the video I uploaded, but the above error occurs during the generation phase. When I re-align it with a different target character it runs fine, should a checking mechanism be added to the project?

wangxiang1230 commented 1 week ago

In stage of Pose alignment, when i run inference.py in my uplated video, it show above error.

Hi,in the stage of pose alignment, you need to run the command like “python run_align_pose.py --ref_name data/images/WOMEN-Blouses_Shirts-id_00004955-01_4_full.jpg --source_video_paths data/videos/source_video.mp4 --saved_pose_dir data/saved_pose/WOMEN-Blouses_Shirts-id_00004955-01_4_full”.

I'am sorry, I've just described it incorrectly. I've done the pose alignment with the video I uploaded, but the above error occurs during the generation phase. When I re-align it with a different target character it runs fine, should a checking mechanism be added to the project?

OK, thanks for your advice. We need to determine what causes this problem, like source image's size is too big? Or something else, do you know what is? Because we haven't had this problem before. Thanks.

shuodehaoa commented 1 week ago

In stage of Pose alignment, when i run inference.py in my uplated video, it show above error.

Hi,in the stage of pose alignment, you need to run the command like “python run_align_pose.py --ref_name data/images/WOMEN-Blouses_Shirts-id_00004955-01_4_full.jpg --source_video_paths data/videos/source_video.mp4 --saved_pose_dir data/saved_pose/WOMEN-Blouses_Shirts-id_00004955-01_4_full”.

I'am sorry, I've just described it incorrectly. I've done the pose alignment with the video I uploaded, but the above error occurs during the generation phase. When I re-align it with a different target character it runs fine, should a checking mechanism be added to the project?

OK, thanks for your advice. We need to determine what causes this problem, like source image's size is too big? Or something else, do you know what is? Because we haven't had this problem before. Thanks.

I didn't find out what the reason was. But I can give you as detailed a description as possible of what I tried for your inference. The source video I uploaded is 720x900 and is about 10 seconds long. The reference photo I uploaded is 720x720. Everything is fine when the poses are aligned. When reasoning, the above error occurs.

By the way, I can confirm that there is nothing wrong with the environment in which the code is running, as I completed the demo test you provided using the same environment.

Interestingly, using the Image I uploaded + the source video in the demo for alignment and generation, everything works fine. Alignment and generation using the video I uploaded + one of the Image in the demo, again everything works fine.

But using my uploaded Image + my uploaded video, the above error occurs.