GPU vs. TPU, A100, V100, T4

[mdlieber99]

Do you have a recommendation for which runtime configuration will produce a batch of images the fastest when running your notebook?

[TheLastBen]

RunPod 3080 and 3090 are the best for inference, colab GPUs aren't that good

[mdlieber99]

So I paid on run pod and I can get to a point where it shows me "connection options" and it gets to a point with the image below but then none of these take me to a place where I can get the standard automatic1111 interface. The first option stays "not ready". the second option takes me to a page where I have no idea how to do anything. If I connect to the web terminal it takes me to a terminal like prompt. How do I get to the interface? And is there a way to get this to use what's in my google drive for models, lora's, etc, or is this a new drive that I need to upload stuff to? Thanks in advance for your help!

[mdlieber99]

Also, while I'm trying to figure that out, is there one of the GPUs on colab that are the best? Seems like A100 followed by V100 followed by T4? Is that right

[TheLastBen]

click on "connect to jupyter lab", make sure you chose the fast stable diffusion template, when you click on connect to jupyter lab, it will take you to jupyter interface, choose the A1111 notebook, it will look similar to the colab notebook.

A100 is way stronger than T4 but 5 times more expensive

[mdlieber99]

Is there a way to connect my google drive so it can grab models from there? Also, when I followed your instructions I was able to get to the automatic1111 start page, but when I ran the start stable diffusion cell, I got this error:

FileNotFoundError Traceback (most recent call last) Cell In[3], line 8 3 Password= "" 5 # Add credentials to your Gradio interface (optional). 6 7 #----------------- ----> 8 configf=sd(User, Password, model) if 'model' in locals() else sd(User, Password, "");import gradio;gradio.close_all() 9 get_ipython().system('python /workspace/sd/stable-diffusion-webui/webui.py $configf')

File /workspace/mainrunpodA1111.py:403, in sd(User, Password, model) 399 auth="" 401 call('wget -q -O /usr/local/lib/python3.10/dist-packages/gradio/blocks.py https://raw.githubusercontent.com/TheLastBen/fast-stable-diffusion/main/AUTOMATIC1111_files/blocks.py', shell=True) --> 403 os.chdir('/workspace/sd/stable-diffusion-webui/modules') 404 call('wget -q -O paths.py https://raw.githubusercontent.com/TheLastBen/fast-stable-diffusion/main/AUTOMATIC1111_files/paths.py', shell=True) 405 call("sed -i 's@/content/gdrive/MyDrive/sd/stablediffusion@/workspace/sd/stablediffusion@' /workspace/sd/stable-diffusion-webui/modules/paths.py", shell=True)

FileNotFoundError: [Errno 2] No such file or directory: '/workspace/sd/stable-diffusion-webui/modules'

[TheLastBen]

did you run all the cells from the top ?

[scorpi0n81]

Do you have a recommendation for which runtime configuration will produce a batch of images the fastest when running your notebook?

I am using V100 and it creates batch of 4 pretty fast. V100 is a good balanced point of return of cost as consumes less compute units per hour and is decent in terms of speed in terms of delivery. A100 is yes faster but the additional per hour compute units dont justify what you get in return.

[mdlieber99]

Thanks for the feedback. In your opinion then, do you get more images per dollar with V100 than A100?

[scorpi0n81]

Thanks for the feedback. In your opinion then, do you get more images per

dollar with V100 than A100?

Not saying it mathematically as the kind of model settings etc.. will influence time per image however, purely in my opinion and experience - yes, V100 hits the sweet spot compared to A100 in terms of more bang for the buck!

[TheLastBen]

yes, if you can get the v100, it's the best option for colab