divamgupta / stable-diffusion-tensorflow

Stable Diffusion in TensorFlow / Keras
Other
1.57k stars 227 forks source link

Rectangular image generation? #30

Closed SidJain1412 closed 1 year ago

SidJain1412 commented 1 year ago

Tried creating a generator where width != height, fails with this error:


ValueError                                Traceback (most recent call last)
[<ipython-input-4-1026659721fc>](https://localhost:8080/#) in <module>
      5     img_height=640,
      6     img_width=340,
----> 7     jit_compile=False,  # You can try True as well (different performance profile)
      8 )

4 frames
[/usr/local/lib/python3.7/dist-packages/stable_diffusion_tf/diffusion_model.py](https://localhost:8080/#) in loop_body_4(itr_4)
    107                     nonlocal x
    108                     b = itr_4
--> 109                     x = ag__.converted_call(ag__.ld(tf).concat, ([ag__.ld(x), ag__.converted_call(ag__.ld(saved_inputs).pop, (), None, fscope)],), dict(axis=(- 1)), fscope)
    110 
    111                     def get_state_5():

ValueError: Exception encountered when calling layer "u_net_model_1" (type UNetModel).

in user code:

    File "/usr/local/lib/python3.7/dist-packages/stable_diffusion_tf/diffusion_model.py", line 216, in call  *
        x = tf.concat([x, saved_inputs.pop()], axis=-1)

    ValueError: Dimension 2 in both shapes must be equal, but are 12 and 11. Shapes are [?,20,12] and [?,20,11]. for '{{node u_net_model_1/concat_3}} = ConcatV2[N=2, T=DT_HALF, Tidx=DT_INT32](u_net_model_1/upsample_2/padded_conv2d_129/conv2d_129/BiasAdd, u_net_model_1/spatial_transformer_18/add, u_net_model_1/concat_3/axis)' with input shapes: [?,20,12,1280], [?,20,11,1280], [] and with computed input tensors: input[2] = <-1>.

Call arguments received by layer "u_net_model_1" (type UNetModel):
  • inputs=['tf.Tensor(shape=(None, 80, 42, 4), dtype=float16)', 'tf.Tensor(shape=(None, 320), dtype=float16)', 'tf.Tensor(shape=(None, 77, 768), dtype=float16)']```

Is it possible to generate rectangular images using this implementation?
kechan commented 1 year ago

i tried this

generator = Text2Image( 
    img_height=512,
    img_width=768,
    jit_compile=False,  # You can try True as well (different performance profile)
)

and that has worked. Just maybe not all aspect ratio works. I picked one that's of standard for a photo.

SidJain1412 commented 1 year ago

@kechan that seems to work for me as well, thank you

matpalm commented 1 year ago

both the height & width need to be multiples of 128

SidJain1412 commented 1 year ago

Thanks @matpalm , couldn't find that mentioned in documentation. Closing this issue