Unable to install Dependencies on Paperspace

[MAI-96]

When trying to run the dependencies block I get the following error:

---

HTTPError Traceback (most recent call last) Cell In[1], line 8 3 force_reinstall= False 5 # Set to true only if you want to install the dependencies again. 6 7 #-------------------- ----> 8 with open('/dev/null', 'w') as devnull:import requests, os, importlib;sc="mainpaperspacev1.py" if os.path.exists('/usr/local/lib/python3.9') else "mainpaperspacev1_311.py";open('/notebooks/mainpaperspacev1.py', 'wb').write(requests.get('https://huggingface.co/datasets/TheLastBen/PPS/raw/main/Scripts/'+sc).content);os.chdir('/notebooks');import mainpaperspacev1;importlib.reload(mainpaperspacev1);from mainpaperspacev1 import *;Deps(force_reinstall)

File /notebooks/mainpaperspacev1.py:58, in Deps(force_reinstall) 56 call('wget -q -i https://raw.githubusercontent.com/TheLastBen/fast-stable-diffusion/main/Dependencies/aptdeps_311.txt', shell=True) 57 call('dpkg -i *.deb', shell=True, stdout=open('/dev/null', 'w')) ---> 58 depsinst("https://huggingface.co/TheLastBen/dependencies/resolve/main/ppsdeps_311.tar.zst", "/deps/ppsdeps_311.tar.zst") 59 call('tar -C / --zstd -xf ppsdeps_311.tar.zst', shell=True, stdout=open('/dev/null', 'w')) 60 os.chdir('/notebooks')

File /notebooks/mainpaperspacev1.py:79, in depsinst(url, dst) 77 file_size = None 78 req = Request(url, headers={"User-Agent": "torch.hub"}) ---> 79 u = urlopen(req) 80 meta = u.info() 81 if hasattr(meta, 'getheaders'):

File /usr/lib/python3.11/urllib/request.py:216, in urlopen(url, data, timeout, cafile, capath, cadefault, context) 214 else: 215 opener = _opener --> 216 return opener.open(url, data, timeout)

File /usr/lib/python3.11/urllib/request.py:525, in OpenerDirector.open(self, fullurl, data, timeout) 523 for processor in self.process_response.get(protocol, []): 524 meth = getattr(processor, meth_name) --> 525 response = meth(req, response) 527 return response

File /usr/lib/python3.11/urllib/request.py:634, in HTTPErrorProcessor.http_response(self, request, response) 631 # According to RFC 2616, "2xx" code indicates that the client's 632 # request was successfully received, understood, and accepted. 633 if not (200 <= code < 300): --> 634 response = self.parent.error( 635 'http', request, response, code, msg, hdrs) 637 return response

File /usr/lib/python3.11/urllib/request.py:563, in OpenerDirector.error(self, proto, args) 561 if http_err: 562 args = (dict, 'default', 'http_error_default') + orig_args --> 563 return self._call_chain(args)

File /usr/lib/python3.11/urllib/request.py:496, in OpenerDirector._call_chain(self, chain, kind, meth_name, args) 494 for handler in handlers: 495 func = getattr(handler, meth_name) --> 496 result = func(args) 497 if result is not None: 498 return result

File /usr/lib/python3.11/urllib/request.py:643, in HTTPDefaultErrorHandler.http_error_default(self, req, fp, code, msg, hdrs) 642 def http_error_default(self, req, fp, code, msg, hdrs): --> 643 raise HTTPError(req.full_url, code, msg, hdrs, fp)

HTTPError: HTTP Error 404: Not Found

Doesn't really matter which of the files I try to install the dependencies from (like using the ConfyUI one vs. the A1111)

[SRagy]

Did you follow the instructions I gave? I said to update the space before doing anything else.

Press the blue text at the bottom where it says update available. Are you able to see something as above and have you pressed it?

[NissanSUTUTU]

Did you follow the instructions I gave? I said to update the space before doing anything else.

Press the blue text at the bottom where it says update available. Are you able to see something as above and have you pressed it?

Yes, i pressed, I thought it didn't appear for me because your screen is light, mine is dark, so I thought the button would be in JupyterLAB. Okay, I pressed it, then I re-uploaded the files that were in the ZIP, I tried to start "Dependencies", this appeared:

[SRagy]

Just run the rest of the notebook anyway.

[NissanSUTUTU]

[SRagy]

Please restart the machine, navigate to the notebook and press run all. Tell me if you get the same error. Also, be aware of the initial message where I described a related error with pytorch lightning.

[NissanSUTUTU]

I restarted the machine, I ran all, the same error happened

[NissanSUTUTU]

pytorch_lightning.utilities.rank_zero I put this on line 20 of the place you mentioned, nothing different happened

[SRagy]

Can you see a line with pytorch_lightning in requirements.txt?

[NissanSUTUTU]

28 line

[SRagy]

Line 20 of the ddpm file should say this

from pytorch_lightning.utilities.rank_zero import rank_zero_only

Is that what you've written, because it looks like you might have

pytorch_lightning.utilities.rank_zero

or

pytorch_lightning.utilities.distributed

[NissanSUTUTU]

Line 20 of the ddpm file should say this

from pytorch_lightning.utilities.rank_zero import rank_zero_only

Is that what you've written, because it looks like you might have

pytorch_lightning.utilities.rank_zero

or

pytorch_lightning.utilities.distributed

[SRagy]

Yes, it's wrong. Please correct it to the first line that begins with from like I said

[NissanSUTUTU]

I put "From". New error:

[SRagy]

Please read the instructions properly. This is the line

from pytorch_lightning.utilities.rank_zero import rank_zero_only

[NissanSUTUTU]

Worked finally!

[Mikeartia]

Please @TheLastBen make it work again, i need it to work, my source of income come from your progam, please

[SRagy]

@Mikeartia Follow the instructions here for a workaround https://github.com/TheLastBen/fast-stable-diffusion/issues/2868#issuecomment-2211953469.

Please follow them carefully - I spent a lot of time trying to help someone earlier when all the information was available in the instructions (not their fault - a language barrier, but still time consuming).

[Mikeartia]

How do i solve this problem?

@Mikeartia Follow the instructions here for a workaround #2868 (comment).

Please follow them carefully - I spent a lot of time trying to help someone earlier when all the information was available in the instructions (not their fault - a language barrier, but still time consuming).

[Archon332]

@Mikeartia Restart the machine.

[Mikeartia]

pytorch_lightning.utilities.rank_zero

Thank you, its installing my fingers is crossed

[Mikeartia]

What happend?

[Mikeartia]

It happend on dependencies but it goes done right after it shows

But i changed it on the line 28 as i was told to

[Mikeartia]

i changed the 28 line to: from pytorch_lightning.utilities.rank_zero import rank_zero_only and i got this error

[SRagy]

Read the linked instructions again please. You're not supposed to change anything in requirements.txt

Also, have you deleted the extensions folder from your sd/stable-diffusion-webui folder? Can you expand the directory and show a screenshot?

[Mikeartia]

I started a new computer from scratch and i got this error on dependencies:

I just ran the notebook and i got this error on SD:

Can you help me? @SRagy

[SRagy]

I have not made the fix for fresh installs - this is stated clearly in the instructions I gave. It's a patch which expects you to have an sd folder already. I might look into doing something more general later. The dependency error is not fatal and things would work as long as you have the sd install; this solution is meant to be a repair not a ground-up install.

If you want to start something from scratch you could try following the automatic1111 linux installation instructions from their repository https://github.com/AUTOMATIC1111/stable-diffusion-webui?tab=readme-ov-file#automatic-installation-on-linux.

I am sorry, but I am no longer going to respond to people who are not carefully reading the instructions I gave. I will instead see if I can make a more foolproof solution because this will be a more efficient use of my time.

[Mikeartia]

I'm sorry @SRagy, i was afraid at the beggining to change anything on my already made SD on papper, i'll take some time and try again later on my own as you told above, thank you for trying to help me

[SRagy]

I understand @Mikeartia. I am not angry or anything, but I think it will be more helpful to everyone if I make an easier to set up fix and deployment instead of trying to fix people's problems individually.

[Archon332]

Talking about fresh install, I deleted old Paperspace notebook and started new from scratch. And SD works well. I just pasted lunk to Ben's PPS, ran second cell of Ben's file bypassing dependencies to install SD folder, moved Ben's python files to "Old notebooks'' folder and started Sragy's file. And also I edited that file with 20th line, just like in instruction.

[Mikeartia]

@Archon332 Do you have discord, i'd like to make a fresh install, but i don't remember how to do it well, could you help me with that?

[aleeepp]

Ben's move is very sad!! I hope you realize the blow you are dealing to us as a community!!!

Where can we go now?

[Wispash]

@Archon332 @SRagy Is there any way to contact you?

[Archon332]

For everyone, who wanted to contact ,my discord, you can PM me: Archon33. Avatar is girl with black hood, black mask, red eyes and brown hair just like here, but another picture. I'd like to NOT use video chat, because I'm bad in english, only with PM.

[davedaverave]

Alright all, I've tried to make these changes as unintrusive as possible in case TheLastBen ever comes back and you decide you want to rely on his configuration again. I'm assuming here that you have everything pretty much set up so you have folders for ComfyUI , sd/stablediffusion and sd/stable-diffusion-webui. If you want an install from scratch, then this solution is not configured for that, but I think you wouldn't be here unless you had established setups you wanted to return to.

Create backups

Begin by creating a folder called old_notebooks. Move the various files you have into this folder such Automatic1111.ipynb and mainpaperspaceA1111.py, and the similar files for ComfyUI

Ensure your space is up to date.

Everything here is set up for Python 3.11. Make sure your space is up to date; if it is not, I believe you should see an update button at the bottom, as in this image (the small blue text saying update available)

Upload files

Upload the files contained in this zip: paperspace.zip. Put them directly in the main notebooks folder where you've just moved the files from, i.e. at the top of the visible directory. Do not create or place in any subfolders.

You should be able to run things as you're used to - it might take a bit longer to install the dependencies, but just a couple of minutes.

A possible error

I had a line of code I needed to change - but don't panic, it's very simple. If you get a fatal error along the lines of ModuleNotFoundError: No module named 'pytorch_lightning.utilities.distributed', then do the following.

Navigate to sd/stablediffusion/ldm/models/diffusion/ddpm.py. In line 20 change pytorch_lightning.utilities.distributed to pytorch_lightning.utilities.rank_zero. You might want to leave yourself a note that you've done this, e.g.

from pytorch_lightning.utilities.rank_zero import rank_zero_only # once was pytorch_lightning.utilities.distributed

If you just receive a warning of the form Pytorch_lightning.distributed not found, attempting pytorch_lightning.rank_zero then this is nothing to worry about and you don't need to do anything.

Notes

I've removed most (but not quite all) calls to TheLastBen's repos to reduce the risk of further breakages. With how things were set up, this means Automatic1111 will no longer update, unless you uncomment the repo() call in the notebook - do so at your own risk. ComfyUI might update, and if some package dependencies change this might cause you to get error messages, but hopefully you'll be able to google fixes for them, because I imagine a few people have issues whenever updates occur anyway.

Let me know if this is now working for you (I made a couple of last minute changes, but hopefully nothing damaging).

Hey, first things first, thank you! I unfortunately can't see the update message in my jupyter noteboow windows... my py is 3.9, i'm trying using comfyui btw...

[SRagy]

@davedaverave Not sure if I'll be able to help with that, but what text do you have down next to where in my screenshot it says update available?

Mine currently says

paperspace/gradient-base:pt211-tf215-cudatk120-py311-20240202

which gives the pytorch and cuda versions and such.

[davedaverave]

@davedaverave Not sure if I'll be able to help with that, but what text do you have down next to where in my screenshot it says update available?

Mine currently says

paperspace/gradient-base:pt211-tf215-cudatk120-py311-20240202

which gives the pytorch and cuda versions and such.

Mine is like that.. did you create a new notebook in a specific way? I'm not sure how to update to new pythong right now... I'm not sure what i'm doing wrong, sorry for the trouble

[SRagy]

I don't think you're doing anything wrong and I'm not sure why it's not showing the update message.

I would try using a different browser just to check (I'm looking at mine in edge). If that doesn't work, you could try updating python manually (google how to do it, you will need to open a terminal from the bar on the left). I'm not confident in this working though, but worth a try. You could also try emailing paperspace support to ask if there's another way to update the page.

Any changes you make to python will be undone when you stop the machine, so don't worry about breaking things. If you figure out what's going on, please come back to tell us so I can update the instructions.

EDIT: also, try disabling ad blockers and such.

[TheLastBen]

The dependencies will be fixed in a few minutes

[dreamlogic-X]

Hey everyone, coming to this thread late but wanted to say thank you @TheLastBen for your ongoing support for the SD community. And @SRagy for stepping in also with the great solution.

It's understandable your frustration. I know a lot of us are frustrated with Paperspace also and the fact that you put in your own time to help us all is really appreciated by all of us. I think it's the type of thing where we'll only say something if things are broken, but we should really show our support more often for you and other members of the community that are helping us all realize our passion.

If ever you have ideas of how we can help you with this project, please do let us know. I know there was some talk above about emailing Paperspace support together, which sounds like a good idea. But it would be good to coordinate what kind of message we are trying to send to them and what we are asking.

Also free feel to post a "Buy me a coffee" and myself and others I'm sure will gladly contribute.

Cheers!

[TheLastBen]

@dreamlogic-X thank you, no need to email paperspace anymore, things are getting solved at the moment.

[MAI-96]

Well I am glad that this somehow got resolved, thank you @SRagy for everything you did while Ben fought with Paperspace (and won apparently? I'm glad that's the case! Although a bit saddened that it took this sort of measure for them to listen to him, to put it mildly.) I just hope theres is no need for this to happen again (either Paperspace learns or they make things work in such a way that we don't have to fight with them over something so fucking basic and simple. I mean they have articles on their website talking about this Github... one would think it would be pretty OBVIOUS that they should make this Notebook official, as a matter of fact I at first thought that it already was and was very confused.)

Okay but all of that is whatever right now I guess, I just wanted to say thank you. As for @TheLastBen if it ever happens again, I hope you know that you have a community around you that is willing to help you!! You just have to ask them.

[TheLastBen]

@MAI-96 Thank you!

[ciwksl]

Hey, thanks for giving a bit more information. I'm sure they have gained a bunch of customers through your notebooks, so I understand how it must be annoying to be completely blanked. As a paying customer, I'd be happy to shoot them an email about this stuff, but I'd like to be clear that I understand it first so I know what to say because I don't really know much about how paperspace is set up.

Is there some sort of option that could make your deployment an official template? Is them not implementing this option making it harder work for you? Causing more bugs? Making setting things up more finnicky somehow?

Also, why break the deployment completely instead of announcing you will not support it any further and removing links to PPS in your github repo? You might have more luck getting people to email pps if you put an explainer in your A1111.ipynb and ComfyUI.ipynb notebooks saying why you no longer plan to support PPS and that it might break in the future because of it.

no such thing as because or what to or know or not etc, say, can say x etc any nmw and any s perfect

[gateway]

anyone having issues as of late getting the domain when running the notebook? seems like the ip is 0.0.0.0 ?

[TheLastBen]

@gateway it has been fixed, provide a screenshot of the error

[gateway]

@gateway it has been fixed, provide a screenshot of the error

@TheLastBen

[TheLastBen]

@gateway you probably didn't pull the updated mainppsComfy_311.py

[gateway]

@gateway you probably didn't pull the updated mainppsComfy_311.py

@SRagy @TheLastBen

Hey I'm not sure what the issue is I have all the latest from https://github.com/SRagy/PPS/tree/main , this only happened a few days ago and nothing was updated so im sorta scratching my head.. also im a dev so I can try to debug but not sure where paperspace tries to assign the domain at launch.. ?

Which repo should be used for the future ? Please and Thank you!

[gateway]

I also just set up a brand new notebook to test it with the sray stuff and still got this.. ? im using the free gpus.. not sure if that matters these days.. but something seems broken with the assignment for dns??

[TheLastBen]

https://github.com/SRagy/PPS is a fork https://github.com/TheLastBen/PPS is the official repo