search

Stability-AI / generative-models

Generative Models by Stability AI
MIT License
23.14k stars 2.56k forks source link

could not broadcast input array from shape (636,732,4) into shape (636,48,4) #337

Closed zoldaten closed 2 months ago

zoldaten commented 3 months ago

hi ! i am trying with python scripts/sampling/simple_video_sample.py --input_path assets/berry.png --version sv3d_u berry but have error (in title):

generative-models/scripts/sampling/simple_video_sample.py", line 155, in sample
    padded_image[
ValueError: could not broadcast input array from shape (636,732,4) into shape (636,48,4)

with test image all OK.

whats wrong ?

merkol commented 3 months ago

same issue here even though I provide an image with resolutions (576, 1024, 3) same as test image. I get broadcasting error

Farenweh commented 2 months ago

I got the very same question and still found no reason or workaround.

Farenweh commented 2 months ago

I've found a workaround, try modifing side_len = ( int(max_size / image_frame_ratio) if image_frame_ratio is not None else in_w ) to side_len = ( int(max_size / image_frame_ratio) if image_frame_ratio is not None else max(in_w, in_h)) ，by applying this workaround, the script could avoid wrong segement mask from simple cv2.threshold( np.array(image.split()[-1]), 0, 255, cv2.THRESH_BINARY causing array be small incorrectly.

HOWEVERE, it's just a workaound which may lead to performance degrading cauz it warns "WARNING: The conditioning frame you provided is not 576x576. This leads to suboptimal performance as model was only trained on 576x576.", thus we may need an elegent fix.

zoldaten commented 2 months ago

I've found a workaround, try modifing side_len = ( int(max_size / image_frame_ratio) if image_frame_ratio is not None else in_w ) to side_len = ( int(max_size / image_frame_ratio) if image_frame_ratio is not None else max(in_w, in_h)) ，by applying this workaround, the script could avoid wrong segement mask from simple cv2.threshold( np.array(image.split()[-1]), 0, 255, cv2.THRESH_BINARY causing array be small incorrectly.

HOWEVERE, it's just a workaound which may lead to performance degrading cauz it warns "WARNING: The conditioning frame you provided is not 576x576. This leads to suboptimal performance as model was only trained on 576x576.", thus we may need an elegent fix.

it`s working -

https://github.com/Stability-AI/generative-models/assets/29411817/c5ee0831-39bd-4529-9e1f-2853eab3d9ca