TheLastBen / fast-stable-diffusion

fast-stable-diffusion + DreamBooth
MIT License
7.51k stars 1.31k forks source link

Unable to install Dependencies on Paperspace #2868

Closed MAI-96 closed 2 months ago

MAI-96 commented 3 months ago

When trying to run the dependencies block I get the following error:


HTTPError Traceback (most recent call last) Cell In[1], line 8 3 force_reinstall= False 5 # Set to true only if you want to install the dependencies again. 6 7 #-------------------- ----> 8 with open('/dev/null', 'w') as devnull:import requests, os, importlib;sc="mainpaperspacev1.py" if os.path.exists('/usr/local/lib/python3.9') else "mainpaperspacev1_311.py";open('/notebooks/mainpaperspacev1.py', 'wb').write(requests.get('https://huggingface.co/datasets/TheLastBen/PPS/raw/main/Scripts/'+sc).content);os.chdir('/notebooks');import mainpaperspacev1;importlib.reload(mainpaperspacev1);from mainpaperspacev1 import *;Deps(force_reinstall)

File /notebooks/mainpaperspacev1.py:58, in Deps(force_reinstall) 56 call('wget -q -i https://raw.githubusercontent.com/TheLastBen/fast-stable-diffusion/main/Dependencies/aptdeps_311.txt', shell=True) 57 call('dpkg -i *.deb', shell=True, stdout=open('/dev/null', 'w')) ---> 58 depsinst("https://huggingface.co/TheLastBen/dependencies/resolve/main/ppsdeps_311.tar.zst", "/deps/ppsdeps_311.tar.zst") 59 call('tar -C / --zstd -xf ppsdeps_311.tar.zst', shell=True, stdout=open('/dev/null', 'w')) 60 os.chdir('/notebooks')

File /notebooks/mainpaperspacev1.py:79, in depsinst(url, dst) 77 file_size = None 78 req = Request(url, headers={"User-Agent": "torch.hub"}) ---> 79 u = urlopen(req) 80 meta = u.info() 81 if hasattr(meta, 'getheaders'):

File /usr/lib/python3.11/urllib/request.py:216, in urlopen(url, data, timeout, cafile, capath, cadefault, context) 214 else: 215 opener = _opener --> 216 return opener.open(url, data, timeout)

File /usr/lib/python3.11/urllib/request.py:525, in OpenerDirector.open(self, fullurl, data, timeout) 523 for processor in self.process_response.get(protocol, []): 524 meth = getattr(processor, meth_name) --> 525 response = meth(req, response) 527 return response

File /usr/lib/python3.11/urllib/request.py:634, in HTTPErrorProcessor.http_response(self, request, response) 631 # According to RFC 2616, "2xx" code indicates that the client's 632 # request was successfully received, understood, and accepted. 633 if not (200 <= code < 300): --> 634 response = self.parent.error( 635 'http', request, response, code, msg, hdrs) 637 return response

File /usr/lib/python3.11/urllib/request.py:563, in OpenerDirector.error(self, proto, args) 561 if http_err: 562 args = (dict, 'default', 'http_error_default') + orig_args --> 563 return self._call_chain(args)

File /usr/lib/python3.11/urllib/request.py:496, in OpenerDirector._call_chain(self, chain, kind, meth_name, args) 494 for handler in handlers: 495 func = getattr(handler, meth_name) --> 496 result = func(args) 497 if result is not None: 498 return result

File /usr/lib/python3.11/urllib/request.py:643, in HTTPDefaultErrorHandler.http_error_default(self, req, fp, code, msg, hdrs) 642 def http_error_default(self, req, fp, code, msg, hdrs): --> 643 raise HTTPError(req.full_url, code, msg, hdrs, fp)

HTTPError: HTTP Error 404: Not Found

Doesn't really matter which of the files I try to install the dependencies from (like using the ConfyUI one vs. the A1111)

TheLastBen commented 3 months ago

you'll have to contact paperspace support

Archon332 commented 3 months ago

This: https://huggingface.co/datasets/TheLastBen/PPS/raw/main/Scripts/'+sc).content);os.chdir('/notebooks');import And this: https://huggingface.co/TheLastBen/dependencies/resolve/main/ppsdeps_311.tar.zst" Links are telling that "Entry not found" while another is active: https://raw.githubusercontent.com/TheLastBen/fast-stable-diffusion/main/Dependencies/aptdeps_311.txt', I don't know, but it may help, probably.

TheLastBen commented 3 months ago

Paperspace will probably drop SD support soon, you'll need to confirm that with them

sliabhmish commented 3 months ago
image

This got deleted 5 hours ago and it appears that the cell trys to pull from it and fails.

Archon332 commented 3 months ago

I send a message to Paperspace support and got this: "Based on the logs you've provided, it appears that you may not have the necessary permissions to download or access the resource.

The HTTP 404 error typically indicates that the requested resource was not found or that access permissions are insufficient. Please ensure that your credentials are correct and that you have proper access to the desired resource.

If you require further assistance or clarification on how to proceed, please feel free to reach out."

sliabhmish commented 3 months ago

now the hugging face dependencies has been redeployed after I made the comment above. Commit history gone. It would appear you've purposefully broken your notebook

sliabhmish commented 3 months ago

and paperspace removed from the readme. Are you removing paperspace support?

TheLastBen commented 3 months ago

it depends on them if they want to keep supporting SD

altccount commented 3 months ago

Hi, I sent them a message and they told me the same as the guy above. I think it's a problem with the notebook.

TheLastBen commented 3 months ago

@altccount it sounds like a chatgpt answer, their support is deserted, I recommend switching to runpod

MAI-96 commented 3 months ago

Sorry to go out on a limb, but honestly this sounds like you are not getting along with Paperspace anymore and have broken their notebook on purpose and instead of just being transparent about it you are passing on the blame to them. You even removed a comment from sliabhmish who probably reached the same conclusion as me.

And I mean what do I know? Maybe they told you to do that, but the way it was done is weird. You could at least be honest instead of dodgy and passive aggressive.

As for Runpod, I prefer Paperspace because I can pay monthly and don't have to do the math about how many hours I'll be running the notebook, but I guess that's just my personal preference and probably not relevant here...

davedaverave commented 3 months ago

Same issue there, it's really unfortunate as i can't economically sustain runpod, i have already a Growth plan with paperspace... i came back to that just for using ComfyUI and make my passion a Job... i'm glad for you work until today, that was the only way i could access such tech..


HTTPError Traceback (most recent call last) Cell In [1], line 8 3 force_reinstall= True 5 # Set to true only if you want to install the dependencies again. 6 7 #-------------------- ----> 8 with open('/dev/null', 'w') as devnull:import requests, os, importlib;open('/notebooks/mainppsComfy.py', 'wb').write(requests.get('https://huggingface.co/datasets/TheLastBen/PPS/raw/main/Scripts/mainppsComfy.py').content);os.chdir('/notebooks');import mainppsComfy;importlib.reload(mainppsComfy);from mainppsComfy import *;Deps(force_reinstall)

File /notebooks/mainppsComfy.py:48, in Deps(force_reinstall) 46 call('wget -q -i https://raw.githubusercontent.com/TheLastBen/fast-stable-diffusion/main/Dependencies/aptdeps.txt', shell=True) 47 call('dpkg -i *.deb', shell=True, stdout=open('/dev/null', 'w')) ---> 48 depsinst("https://huggingface.co/TheLastBen/dependencies/resolve/main/ppsdeps.tar.zst", "/deps/ppsdeps.tar.zst") 49 call('tar -C / --zstd -xf ppsdeps.tar.zst', shell=True, stdout=open('/dev/null', 'w')) 50 call("sed -i 's@~/.cache@/notebooks/cache@' /usr/local/lib/python3.9/dist-packages/transformers/utils/hub.py", shell=True)

File /notebooks/mainppsComfy.py:69, in depsinst(url, dst) 67 file_size = None 68 req = Request(url, headers={"User-Agent": "torch.hub"}) ---> 69 u = urlopen(req) 70 meta = u.info() 71 if hasattr(meta, 'getheaders'):

File /usr/lib/python3.9/urllib/request.py:214, in urlopen(url, data, timeout, cafile, capath, cadefault, context) 212 else: 213 opener = _opener --> 214 return opener.open(url, data, timeout)

File /usr/lib/python3.9/urllib/request.py:523, in OpenerDirector.open(self, fullurl, data, timeout) 521 for processor in self.process_response.get(protocol, []): 522 meth = getattr(processor, meth_name) --> 523 response = meth(req, response) 525 return response

File /usr/lib/python3.9/urllib/request.py:632, in HTTPErrorProcessor.http_response(self, request, response) 629 # According to RFC 2616, "2xx" code indicates that the client's 630 # request was successfully received, understood, and accepted. 631 if not (200 <= code < 300): --> 632 response = self.parent.error( 633 'http', request, response, code, msg, hdrs) 635 return response

File /usr/lib/python3.9/urllib/request.py:561, in OpenerDirector.error(self, proto, args) 559 if http_err: 560 args = (dict, 'default', 'http_error_default') + orig_args --> 561 return self._call_chain(args)

File /usr/lib/python3.9/urllib/request.py:494, in OpenerDirector._call_chain(self, chain, kind, meth_name, args) 492 for handler in handlers: 493 func = getattr(handler, meth_name) --> 494 result = func(args) 495 if result is not None: 496 return result

File /usr/lib/python3.9/urllib/request.py:641, in HTTPDefaultErrorHandler.http_error_default(self, req, fp, code, msg, hdrs) 640 def http_error_default(self, req, fp, code, msg, hdrs): --> 641 raise HTTPError(req.full_url, code, msg, hdrs, fp)

HTTPError: HTTP Error 404: Not Found

TheLastBen commented 3 months ago

Same issue there, it's really unfortunate as i can't economically sustain runpod, i have already a Growth plan with paperspace

it's up to paperspace if they want to continue supporting SD, I've been trying to reach them for months, there is no one there, it's like it's run by machines, if you can actually get hold of someone there that is not an LLM auto answer, that would be a miracle.

Archon332 commented 3 months ago

Ben, can you try to restore the missing dependencies on HuggingFace. Missed ones are links 1 and 3 from error code. Probably, it will help.

SRagy commented 3 months ago

It's clear that paperspace have done something which has pissed you off, but I'm not sure why you're being so opaque with us about it. You have plenty of people who have good reason to be on your side since we appreciate that you've spent some effort to make A1111 and ComfyUI easily accessible, but instead you're asking people to contact PPS support when you've removed files which the deployment depends on from your HF repo.

If you have good reason to stop supporting PPS (like if they're hard to work with so it's too much effort) then that's fine, but why not be clear about it instead of sending people off on a wild goose chase?

TheLastBen commented 3 months ago

@SRagy they are unavailable, not a single one of them bothered to answer my request about actually making the access to the notebooks easier, been trying to reach them for months, they simply choose not to even respond with an automated answer, maybe actual customers complaints will wake them up.

The notebooks can always be accessed trough Runpod (who actually bothered to make the template an official one)

SRagy commented 3 months ago

Hey, thanks for giving a bit more information. I'm sure they have gained a bunch of customers through your notebooks, so I understand how it must be annoying to be completely blanked. As a paying customer, I'd be happy to shoot them an email about this stuff, but I'd like to be clear that I understand it first so I know what to say because I don't really know much about how paperspace is set up.

Is there some sort of option that could make your deployment an official template? Is them not implementing this option making it harder work for you? Causing more bugs? Making setting things up more finnicky somehow?

Also, why break the deployment completely instead of announcing you will not support it any further and removing links to PPS in your github repo? You might have more luck getting people to email pps if you put an explainer in your A1111.ipynb and ComfyUI.ipynb notebooks saying why you no longer plan to support PPS and that it might break in the future because of it.

davedaverave commented 3 months ago

I would contribute too, what should we ask them for? How can we make them implement your notebook? Having a message or template to send them would help us, i think...

I as well as other customer wouldn't use pay for hour or usage services, so i would complain that to their customer support if that could help you ..

TheLastBen commented 3 months ago

@davedaverave Simply that they make this repo as the official PPS Gradient template for stable diffusion instead of an old rusty non-working code that they had for 2 years now.

Archon332 commented 3 months ago

I agree with the idea of ​​writing en masse to their support service to support you. Just let us know. But from your side, I cordially request that you restore the files on HuggingFace.

MAI-96 commented 3 months ago

I would have also agreed to contacting them, I have been paying their service for a year at this point (I used to use Voldemort's "Fast Stable Diffusion" before, I had to work shop it myself every time it stopped working but at this point SD has changed so much I can no longer make it work so I had to move to yours) and I totally agree that the one template they have available is a piece of crap. I think having a unified message might also work better than the whole messy confused message I ended up sending them yesterday.

As far as the HuggingFace Links being restored and such, I would obviously appreciate that too, I have a long weekend for the first time in a while and was looking forward to playing around with SD and maybe even training some new models using your notebook. I don't get a lot of free weekends like these anymore. So I would be really grateful if I got that chance. That said this is up to you and not me, I just ask for honesty and communication so that we at least have the chance to attempt to contact Paperspace.

Maybe we can mass report their current notebook and remind them that it doesn't even work properly and they need a better one? (And link to yours as that better one?)

RONNYKHALIL commented 3 months ago

@davedaverave Simply that they make this repo as the official PPS Gradient template for stable diffusion instead of an old rusty non-working code that they had for 2 years now.

I will definitely share with them. I've been a long time customer (often spending double or triple the normal Growth subscription b/c of storage overages) and only for the purpose of using your repo. Paperspace is definitely a world of pain compared to Runpod (which I'm also subscribed and use daily), but the benefit being getting to run A100s for 6 hour chunks or at least A6000s. This is the last straw. If they don't make your repo official soon I'm out for good. Yes Runpod is more expensive but the amount of headaches, time wasted trying workaround, and steady stream of system outages is just not worth the money being saved. Will be sure to let them know

SRagy commented 3 months ago

Unfortunately, because of the decision to intentionally break things, it's a bit difficult to know what to say in an email to paperspace, "hey, please make this presently broken deployment your default, thanks". If anything, it makes it easy for them to justify the decision of not doing so. If you really wanted to put pressure on paperspace this was a strategic error.

Like I said earlier - there's plenty of good will here towards you because a lot of us have found the your stuff very useful; had you taken a more measured approach it would have been easier to leverage this good will in a more productive way. This sort of move just risks evaporating that good will, and reduces any leverage we can place because we can't even point to a working template.

I'm not just saying this because I'm desperate to get things working - I've managed to wrangle things so that I've got things running with just a few hiccups (which I'm sure I'll also be able to get rid of soon).

MAI-96 commented 3 months ago

SRagy

Unfortunately, because of the decision to intentionally break things, it's a bit difficult to know what to say in an email to paperspace, "hey, please make this presently broken deployment your default, thanks". If anything, it makes it easy for them to justify the decision of not doing so. If you really wanted to put pressure on paperspace this was a strategic error.

I gotta agree with you on your comment, because of how it happened and how I reached out to them earlier it's made it hard to know what to say now... It's why I have kind of been waiting to see if something happens or an official statement is given that I can point to.

I'm not just saying this because I'm desperate to get things working - I've managed to wrangle things so that I've got things running with just a few hiccups (which I'm sure I'll also be able to get rid of soon).

Do you think you could share your solution? (I know a bit of coding, I'm am honestly not knowledgeable enough to figure this one out, but if you'd rather keep it private I understand.)

In the meantime I have basically just given up on using SD any time soon.

43rnb7 commented 3 months ago

I want to say "当了婊子,又立牌坊"

NissanSUTUTU commented 3 months ago

@MAI-96 This is extremely bad for me and the community, I depended on it to generate income and to save money to buy a better computer, I could make a lot of average art and get rich, but I focused on high quality art on Stable Diffusion, and that I brought a lot of fans who love what I do, and I won't do this forever, I want to work with 3D, but there are people who still like what I do in AI to this day, I've been a Paperspace subscriber for 1 year, paying USD 75 per month, I don't know the reason for this change, but it was poorly planned, without prior notice, totally disorganized, this should have happened at least 1 month in advance. Please TheLastBen, resolve this however you can, I pay a lot for this and there are many people like me who depend on this to work. I'll send a message to Paperspace

SRagy commented 3 months ago

@MAI-96, @RONNYKHALIL, @sliabhmish, @davedaverave, @43rnb7, @Archon332, @altccount, @NissanSUTUTU

I've managed to fix things for myself. I'll try to package things in an easy-to-use way then post here later. I can't guarantee that what has worked for me will work for you, but it might at least get you close enough that it's easy to filter through any remaining dependency issues through the error messages you obtain.

If there is an issue with me posting here, then I'll post it to my github instead and you can check it there. I'm a bit busy with other stuff at the moment but will hopefully be back in a couple of hours.

Archon332 commented 3 months ago

@SRagy Take your time, do as more comfortable for you. I feel like a drug addict during withdrawal without using SD and you probably my savior.

MAI-96 commented 3 months ago

@SRagy Thank you for helping out the rest of us!

SRagy commented 3 months ago

Alright all, I've tried to make these changes as unintrusive as possible in case TheLastBen ever comes back and you decide you want to rely on his configuration again. I'm assuming here that you have everything pretty much set up so you have folders for ComfyUI , sd/stablediffusion and sd/stable-diffusion-webui. If you want an install from scratch, then this solution is not configured for that, but I think you wouldn't be here unless you had established setups you wanted to return to.

Create backups

Begin by creating a folder called old_notebooks. Move the various files you have into this folder such Automatic1111.ipynb and mainpaperspaceA1111.py, and the similar files for ComfyUI

Ensure your space is up to date.

Everything here is set up for Python 3.11. Make sure your space is up to date; if it is not, I believe you should see an update button at the bottom, as in this image (the small blue text saying update available)

image

Upload files

Upload the files contained in this zip: paperspace.zip. Put them directly in the main notebooks folder where you've just moved the files from, i.e. at the top of the visible directory. Do not create or place in any subfolders.

You should be able to run things as you're used to - it might take a bit longer to install the dependencies, but just a couple of minutes.

A possible error

I had a line of code I needed to change - but don't panic, it's very simple. If you get a fatal error along the lines of ModuleNotFoundError: No module named 'pytorch_lightning.utilities.distributed', then do the following.

Navigate to sd/stablediffusion/ldm/models/diffusion/ddpm.py. In line 20 change pytorch_lightning.utilities.distributed to pytorch_lightning.utilities.rank_zero. You might want to leave yourself a note that you've done this, e.g.

from pytorch_lightning.utilities.rank_zero import rank_zero_only # once was pytorch_lightning.utilities.distributed

If you just receive a warning of the form Pytorch_lightning.distributed not found, attempting pytorch_lightning.rank_zero then this is nothing to worry about and you don't need to do anything.

Notes

I've removed most (but not quite all) calls to TheLastBen's repos to reduce the risk of further breakages. With how things were set up, this means Automatic1111 will no longer update, unless you uncomment the repo() call in the notebook - do so at your own risk. ComfyUI might update, and if some package dependencies change this might cause you to get error messages, but hopefully you'll be able to google fixes for them, because I imagine a few people have issues whenever updates occur anyway.

Let me know if this is now working for you (I made a couple of last minute changes, but hopefully nothing damaging).

Archon332 commented 3 months ago

And I deleted notebook already( Will try.

Archon332 commented 3 months ago

@SRagy It operational! image I ask you if you can to create repository so we will be able to contact you if something happens. Also, even with problem is resolved, I ask everyone to be ready to mail Paperspace for helping Ben.

SRagy commented 3 months ago

I ask you if you can to create repository so we will be able to contact you if something happens. Also, even with problem is resolved, I ask everyone to be ready to mail Paperspace for helping Ben.

I've forked the repository here: https://github.com/SRagy/PPS. If you need to contact me feel free to do so there. I have not yet updated the repository and may not do so, so for now just consider it an area to troubleshoot this workaround.

NissanSUTUTU commented 3 months ago

Certo, pessoal, tentei fazer essas alterações ou menos intrusivas possíveis, caso o TheLastBen volte e vocês decidam que querem confiar na configuração dele novamente. Estou sugerindo aqui que vocês têm tudo praticamente configurado, então vocês têm pastas para ComfyUI, sd/stablediffusion e sd/stable-diffusion-webui. Se você quiser uma instalação do zero, então essa solução não está configurada para isso, mas acho que vocês não estariam aqui a menos que tivessem configurações estabelecidas para que possamos retornar.

Criar backups

Comece criando uma pasta chamada old_notebooks. Mova os vários arquivos que você tem para essa pasta, como Automatic1111.ipynb e mainpaperspaceA1111.py, e os arquivos semelhantes ao ComfyUI

Certifique-se de que seu espaço esteja atualizado.

Tudo aqui está configurado para Python 3.11. Certifique-se de que seu espaço esteja atualizado; se não estiver, acredite que você deve ver um botão de atualização na parte inferior, como nesta imagem (o pequeno texto azul dizendo atualização disponível)

imagem

Fazer upload de arquivos

Carregue os arquivos contidos neste zip: paperspace.zip . Coloque-os diretamente na pasta principal dos notebooks de onde você acabou de mover os arquivos. Você deve conseguir executar as coisas como está acostumado - pode levar um pouco mais de tempo para instalar as dependências, mas apenas alguns minutos.

Um possível erro

Eu tinha uma linha de código que poderia mudar - mas não entre em pânico, é muito simples. Se você receber um erro fatal nas linhas de ModuleNotFoundError: No module named 'pytorch_lightning.utilities.distributed', faça o seguinte.

Navegue até sd/stablediffusion/ldm/models/diffusion/ddpm.py. Na linha 20, altere pytorch_lightning.utilities.distributedpara pytorch_lightning.utilities.rank_zero. Você pode querer deixar uma nota para si mesmo de que fez isso, por exemplo

from pytorch_lightning.utilities.rank_zero import rank_zero_only # once was pytorch_lightning.utilities.distributed

Se você receber apenas um aviso do formulário Pytorch_lightning.distributed not found, attempting pytorch_lightning.rank_zero, não há nada com que se preocupar e você não precisa fazer nada.

Notas

Removi a maioria (mas não todas) das chamadas para os repositórios do TheLastBen para reduzir o risco de mais quebras. Com a forma como as coisas foram configuradas, isso significa que o Automatic1111 não será mais atualizado, a menos que você descole a repo()chamada no notebook - faça isso pela sua conta e risco. O ComfyUI pode atualizar, e se algumas dependências do pacote mudarem, isso pode fazer com que você receba mensagens de erro, mas espero que você consiga pesquisar no Google as correções para elas, porque imagino que algumas pessoas tenham problemas sempre que as atualizações ocorrem de de qualquer maneira.

Deixe-me saber se isso está acontecendo para você agora (faça algumas alterações de última hora, mas espero que nada prejudique).

I didn't understand exactly what to do, can you explain it to me again?

NissanSUTUTU commented 3 months ago

@SRagy Hello, I tried to solve it with the tips above, I couldn't, I think I did something wrong, could you help me fix my notebook so it works again?

amit4aws commented 3 months ago

@SRagy SDXL-LoRA-PPS is downloading the file sdxllorapps.py which still has reference to huggingface just like thelastben and it is erroring out exactly at the same location how is it working for you?

` 7 #-------------------- ----> 8 with open('/dev/null', 'w') as devnull:import requests, os, importlib;sc="sdxllorapps.py" if os.path.exists('/usr/local/lib/python3.9') else "sdxllorapps_311.py";open('/notebooks/sdxllorapps.py', 'wb').write(requests.get('https://huggingface.co/datasets/TheLastBen/PPS/raw/main/Scripts/'+sc).content);os.chdir('/notebooks');import sdxllorapps;importlib.reload(sdxllorapps);from sdxllorapps import *;Deps(force_reinstall)

File /notebooks/sdxllorapps.py:62, in Deps(force_reinstall) 60 call('wget -q -i https://raw.githubusercontent.com/TheLastBen/fast-stable-diffusion/main/Dependencies/aptdeps_311.txt', shell=True) 61 call('dpkg -i *.deb', shell=True, stdout=open('/dev/null', 'w')) ---> 62 depsinst("https://huggingface.co/TheLastBen/dependencies/resolve/main/ppsdeps_311.tar.zst", "/deps/ppsdeps_311.tar.zst") 63 call('tar -C / --zstd -xf ppsdeps_311.tar.zst', shell=True, stdout=open('/dev/null', 'w')) 64 os.chdir('/notebooks')`

NissanSUTUTU commented 3 months ago

image @SRagy Is this correct? I can't start the SD. I dragged the files contained in the ZIP archive to my notebooks folder

NissanSUTUTU commented 3 months ago

image This error occurs when trying to start "Dependencies" @Archon332 @SRagy

amit4aws commented 3 months ago

@NissanSUTUTU first run install for requirments.txt and then execute the cells of notebook. If the depenencies are already installed the script in the notebook will not try to pull it again from huggingface

SRagy commented 3 months ago

@NissanSUTUTU first run install for requirments.txt and then execute the cells of notebook. If the depenencies are already installed the script in the notebook will not try to pull it again from huggingface.

Did this solve your problem for SDXL-LoRA-PPS? I admit I didn't look at any of the training code yet and I don't seem to have that file. If it didn't work, I would suggest that you can run one of the other files first to get the dependencies installed and then you might be able to omit the running the dependencies installation cell entirely in that notebook.

@NissanSUTUTU What happens if you start a new machine and run the A1111 or ComfyUI files? Do those work?

I'll be back later when I'm less busy, and once again if this thread becomes an issue then I'll take things to the forked repositories on my profile.

NissanSUTUTU commented 3 months ago

@NissanSUTUTU first run install for requirments.txt and then execute the cells of notebook. If the depenencies are already installed the script in the notebook will not try to pull it again from huggingface

Where can I run install for "requirements.txt"? I don't know where to do this

image

SRagy commented 3 months ago

@NissanSUTUTU you don't want things in a folder called notebooks which you've made, you want things at the top of the visible directory (so in the same section where you have spandrel.txt). Try putting everything in there and then just running the A1111 notebook and tell me what happens.

DCMartinec commented 3 months ago

Canceled paperspace, not because of this, but because of the huge money waste their services are. Unable to get on any free a100 instances, and the free a6000 instances have been performing poorly. They have constant issues with gradient "breaking" with no commitment to uptime. Going back to Colab for the time being, until I can find something better. Runpod is too expensive for a hobby.

NissanSUTUTU commented 3 months ago

Cancelei o paperspace, não por isso, mas por causa do enorme desperdício de dinheiro que seus serviços são. Não é possível obter nenhuma instância a100 gratuita, e as instâncias a6000 gratuitas têm tido um desempenho ruim. Eles têm problemas constantes com "quebra" de gradiente sem compromisso com o tempo de atividade. Voltando ao Colab por enquanto, até que eu possa encontrar algo melhor. O Runpod é muito caro para um hobby.

Canceled paperspace, not because of this, but because of the huge money waste their services are. Unable to get on any free a100 instances, and the free a6000 instances have been performing poorly. They have constant issues with gradient "breaking" with no commitment to uptime. Going back to Colab for the time being, until I can find something better. Runpod is too expensive for a hobby.

In 1 year, I used about 50% of the time with A6000, and the other 50% with A100-80G, I had few problems, nothing as big as this recent one

NissanSUTUTU commented 3 months ago

@NissanSUTUTU you don't want things in a folder called notebooks which you've made, you want things at the top of the visible directory (so in the same section where you have spandrel.txt). Try putting everything in there and then just running the A1111 notebook and tell me what happens.

image This error happened, I put all the files in that ZIP file outside the folder

SRagy commented 3 months ago

This seems to be an issue with the xformers installation. I can look into it more later, but a couple of things to check:

  1. Start the machine again from scratch.
  2. Make sure that the requirements.txt file is definitely the one that was in the zip
  3. Run your Automatic1111 or comfyUI ipynb

If it still fails then

  1. Start the machine from scratch
  2. Delete the line which says xformers from requirements.txt for now.
  3. Run things without the xformers install. This could have performance impact.

There is some discussion of this issue here: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/2047, although I'm not sure if all the details are the same because it was from a while back.

NissanSUTUTU commented 3 months ago

This seems to be an issue with the xformers installation. I can look into it more later, but a couple of things to check:

  1. Start the machine again from scratch.
  2. Make sure that the requirements.txt file is definitely the one that was in the zip
  3. Run your Automatic1111 or comfyUI ipynb

If it still fails then

  1. Start the machine from scratch
  2. Delete the line which says xformers from requirements.txt for now.
  3. Run things without the xformers install. This could have performance impact.

There is some discussion of this issue here: AUTOMATIC1111/stable-diffusion-webui#2047, although I'm not sure if all the details are the same because it was from a while back.

image I deleted "xformers", restarted the virtual machine, this happened

SRagy commented 3 months ago

Open up a terminal from the bar on the far left. Then type pip --version and tell me what it says (you want to be sure you've updated the machine to use python 3.11). I will be able to look more in a few hours but am currently on my phone not my computer.

You can also try to follow the error message to set the packages in requirements.txt to the versions it suggests. e.g. ,pydantic==1.9.0 but it's up to you if you want to learn how to do that for yourself. I think it's a good idea.

NissanSUTUTU commented 3 months ago

Open up a terminal from the bar on the far left. Then type pip --version and tell me what it says (you want to be sure you've updated the machine to use python 3.11). I will be able to look more in a few hours but am currently on my phone not my computer.

You can also try to follow the error message to set the packages in requirements.txt to the versions it suggests. e.g. ,pydantic==1.9.0 but it's up to you if you want to learn how to do that for yourself. I think it's a good idea.

python 3.9

NissanSUTUTU commented 3 months ago

Some people managed to get the AI ​​to work again, I still haven't managed it