Running on Multiple GPUs

[jacobkreider]

To run on several GPUs, do I set a setting within stable_diffusion_tf, or do I set that in tensorflow/keras and then run the stable_d code?

[matpalm]

you need to config the corresponding keras distribution strategy.

e.g. see this example for TPUs https://colab.research.google.com/drive/17zQOm_2Iu6pcP8otT-v6rx0D-pKgfaLm?usp=sharing

[jacobkreider]

Thanks, got that to work. One more quick related before I close out:

If I run a batch_size of, say 20, I run out of resources. I assume because it holds all arrays in memory until the batches all finished and I save them out or display them and the script closes.

I can get around this by running a batch_size = 1 20x, of course.

Is there any functional difference in output between the two? (I'm guessing not?)

[matpalm]

it's almost the same, but as the code it, not exacly (because of the way the latent is seeded). see https://github.com/divamgupta/stable-diffusion-tensorflow/pull/32 for a way to decouple the latent seeding from the batch size

[jacobkreider]

Yep, after some experimentation I see that. Thanks for the link. Closing out.

[miguelCalado]

Hi, sorry to reopen this. I'm trying to run the stable diffusion on 2xA6000.

Below is the code:

import tensorflow as tf
from stable_diffusion_tf.stable_diffusion import Text2Image
from PIL import Image

strategy = tf.distribute.MirroredStrategy(["GPU:0", "GPU:1"])
generator = Text2Image( 
    img_height=512,
    img_width=512,
    jit_compile=False,  # You can try True as well (different performance profile)
)

img = generator.generate(
    "kermit the frog with wings goes to heaven. fantasy, intricate, elegant, highly detailed, digital painting, artstation, concept art, matte, sharp focus, illustration.",
    num_steps=50, 
    unconditional_guidance_scale=7,
    temperature=1,
    batch_size=15,
)

It does appear that only one GPU is running (print below). Additionally, the time that it takes to run on a single GPU is the same that it takes to run on two. Is it supposed to or am I doing something wrong?

[miguelCalado]

Btw here's Kermit with wings in heaven, according to stable diffusion :D