bbox without range check

[ljx-s]

python -m scripts.inference --inference_config results/musetalk_test_result.yaml

video in 25.0 FPS, audio idx in 50FPS extracting landmarks...time consuming reading images... 100%|████████████████████████████████████████████████████████████████████| 250/250 [00:01<00:00, 207.18it/s]get key_landmark and face bounding boxes with the default value 100%|█████████████████████████████████████████████████████████████████████| 250/250 [00:13<00:00, 19.16it/s]****bbox_shift parameter adjustment** Total frame:「250」 Manually adjust range : [ -16~16 ] , the current value: 0

---

(61, 0, 277, 308) (61, -1, 277, 309) Traceback (most recent call last): File "/envs/musetalk/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/envs/musetalk/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/MuseTalk/scripts/inference.py", line 153, in main(args) File "/envs/musetalk/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "/MuseTalk/scripts/inference.py", line 86, in main crop_frame = cv2.resize(crop_frame,(256,256),interpolation = cv2.INTER_LANCZOS4) cv2.error: OpenCV(4.9.0) /io/opencv/modules/imgproc/src/resize.cpp:4152: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

I print bbox in inference and find that the negtive bbox index would cause this bug.

[itechmusic]

Would you mind sharing the video and audio you used?

[itechmusic]

Hello,

To address the issue, you can incorporate the following line of code at https://github.com/TMElyralab/MuseTalk/blob/main/musetalk/utils/preprocessing.py#L122:

upper_bond = max(0, upper_bond)

This adjustment will ensure that the 'upper_bond' value never falls below zero, effectively preventing bounding box from exceeding its limits when the face is positioned too close to the image's edge.

[ljx-s]

Would you mind sharing the video and audio you used?

I'm sorry I couldn't share that with you. Instead, I can provide the single frame that led to this mistake. I hope this will be helpful for you.