Closed Sh4d0wHunt3rX closed 1 year ago
This is an issue with Auto1111 that needs to be fixed. There is a PR you can cherry-pick until it maybe gets merged.
https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/8118
Every time I update this extension, everything breaks. EVERY TIME. I spend more time fixing issues than I do using the extension. Is there a standalone that isn't a collab with a GUI? I know all of this is free and people work hard on it, just so frustrated, I need to learn to not hit the update button.
Apologize for getting upset @d8ahazard I'm just a novice stumbling in the dark with how all this works on the backend. I realize it's not your issue and waiting for auto on this one. Was just a nightmare in Jan trying to diagnose, learn, and correct all the cuda/torch related stuff to get it working well again. Realize this is all cutting edge and I need to not forget that when things break.
THIS IS NOT A SOLUTION:
Solution in this thread:
https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/8281#issuecomment-1458305713
THIS IS THE SOLUTION: https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/8118
The problem appears to be with gitpython=3.1.31 need to roll back to gitpython=3.1.27
The problem appears to be with gitpython~=3.1.27 being wildly insecure and needing to be updated to 3.1.31, and @Auto1111 not allowing anybody else to merge PR's, even if they provide updates to account for massive security issues.
Every time I update this extension, everything breaks. EVERY TIME. I spend more time fixing issues than I do using the extension. Is there a standalone that isn't a collab with a GUI? I know all of this is free and people work hard on it, just so frustrated, I need to learn to not hit the update button.
Let's talk about this for a moment. Do you know why everything breaks every time you update this extension? Honestly?
Because the extension system in Auto1111 is complete and utter garbage in terms of how it functions.
Each extension can install it's own requirements with no restrictions, and they do so after the main app requirements are installed.
Each extension can monkey-patch and change any other existing code or variable anywhere else in the application with zero restrictions.
Additionally, each extension is allowed unfettered access to any method anywhere else in the application...which maybe sounds neat on the surface, but actually means that any time somebody goes and merges a random PR to the MAIN branch of the auto1111 all willy-nilly like, well, it has the potential to fuck up any extension that could be using those methods.
So, while it's sure allowed for a massive growth in the extensions ecosystem...it's also like the wild wild west in terms of considerations given to extension developers. If Auto1111 pushes some shit that breaks an extension...oh well?!
As such, I've adopted the same mentality. While I'm not deliberately trying to break anything, I also don't really give a flying fuck if me patching an issue causes an underlying issue with the main application to be highlighted.
In this case, there has been a has been a "High Severity" Github Security Advisory for the GitPython library used by Auto1111 published since December 5 of last year...and Auto1111 hasn't done anything about it.
https://github.com/advisories/GHSA-hcpj-qp55-gfph
So, I did. I bumped the version of GitPython that ships with my extension because I use it in my extension, and while it's a small attack vector, it's still one that can be exposed on public webservers...so I'm just not going to do it.
And while I'm really getting into the weeds with my issues with the Auto repo...I suggested months and months ago that the process for developing that application should be changed so that there is a dev and master branch, which would eliminate a lot of the issues extension developers face - which is "shit just changed and now I woke up to a hundred messages that my extension is broken".
I could test my extension on the latest dev branch of Auto1111, verify that it still works with forthcoming changes, and have nearly zero headaches in this department. Instead, it's been a regular nightmare for months now.
Additionally, as you can see if you look at the PR above with the proper fix, Auto1111 has been less and less active in this massively used project...but has also not given ANYBODY else permission to merge changes. Meaning, the end result is that while this could have been fixed a day or so after being reported...instead...there is only one person who can fix it properly, and they are nowhere to be found ATM.
So...with all of this in mind...imagine my continued frustration. Imagine there being proper channels for reporting and fixing issues with the main application; a team of collaborators trusted enough to fix breaking issues in a reasonable amount of time; or some semblance of a proper development cycle where changes are tested before being promoted to EVERYBODY.
Hell, imagine an application that was actually designed to have an extension system. Think about it! A proper sandboxed implementation that doesn't let the extensions break the effing app, and where some malicious developer would actually have to work to release an extension that completely hoses a person's OS. Because the current one has none of that...which is actually kind of terrifying.
With all that said - YES - I am working on a stand-alone frontend designed to completely separate myself from Auto1111, plus remedy all of the issues I've described above. Progress has been slow, as I'm actually trying to plan everything in advance, versus just writing code and then hacking in ideas later on as they come to me. I'm also completely ditching Gradio and writing my own user-interface from scratch, which makes it even more tedious.
But, eventually...I won't have to have weekly conversations like this one, because the application will be my own and I'll have nobody else to blame but myself if things are done half-assed.
Apologize for getting upset @d8ahazard I'm just a novice stumbling in the dark with how all this works on the backend. I realize it's not your issue and waiting for auto on this one. Was just a nightmare in Jan trying to diagnose, learn, and correct all the cuda/torch related stuff to get it working well again. Realize this is all cutting edge and I need to not forget that when things break.
It's cool. See above. I'm even more frustrated than you.
I appreciate the long response and read it in it's entirety. I hope it was good to get out as it was for me to gain a better understanding of what's up.
Is it not possible to just branch off on a new fork or is that considered bad practice to sidestep auto? I don't know his mindset in the least, but it's unfortunate he's not around much having sole discretion on merging PRs especially when his UI seems to be the most popular and with the most features.
I run multiple installations but only one dreambooth. Time to spin up a second.
As it were, sounds exciting that there's a standalone in the works and ditching gradio to boot. I've not been too fond of it's shortcomings such as lacking a basic eraser for inpainting. Usually just use photoshop for more precise masking anyway. Don't know if it's a coding thing or a gradio thing the way updating lists are handled when merging ckpts or training models, why they don't just appear without needing to push refresh.
I appreciate your efforts. I prefer sticking with dreambooth. I know there's other options out there, everywhere? I think it's called. But I've not been motivated to relearn basics to get started.
I'm just trying to get training working now on your plugin. My results are damaged compared to prior. Dunno if I just need to role cuda and torch and xformers back again like I was doing Jan/Feb.
d8 that sounds bad ass. I cant wait to see it! Messing with Auto1111 has been a frustrating mess. Its a fun app, but the weekly headaches...
I mean. I get your frustration. But your main points seem to be that:
The Automatic1111 extension system is cobbled together and lets any extension developer completely jack with the underlying platform in a way that can screw with other extensions’ code.
The Automatic1111 Python library is an insecure dumpster fire. (Which I’d agree with). So you stopped using it in your extension and installed another. Which breaks everything for everyone else but hey, see point 1. You’re right, they’re wrong, if they don’t want their car door dinged they shouldn’t have parked so close.
Which from a point of developer vs developer I understand — the logic is sound. But from the end user perspective, you’ve essentially ended your extension.
The end users mostly seem to be giving your extension a wide berth. Remember; they don’t care about palace intrigue or coder wang fights — from their perspective, it worked great at first and then it kept getting “updates” that made users feel unable to reproduce the good results they got from early versions. And then it started getting “updates” that broke core functionality. And then it started getting “updates” that broke the core functionality of all of the other extensions so that now people are resorting to batch commands and power shell scripts to auto update their extensions. You may feel that this is an unfair characterization. The end users don’t care, they just see what they see and move on.
Most of the YouTube tuts out there and reddit threads now point people to Koya_SS and tell people to avoid your extension. Users really don’t have patience for this kind of thing when there are alternatives. I’m guessing the train has left the station on this whole project and you’re not on it. I see you’re now making your own train with blackjack and hookers that completely abandons gradio and A111 and I genuinely hope you are successful.
Auto also shouldn't be the only one allowed to approve merge requests if he is gonna disappear for 3+ weeks when his webUI is probably the most popular in the world for SD. Two-way street.
Also, can't claim it's completely his anymore anyway, I could guess at a percentage breakdown but there's been a ton of community contribution. The fact people make extensions for it plus bug fixes and what not. In some ways extensions people develop and add for webUI are probably carrying the whole thing now. How much is still auto's?
I've nothing personal against him at all, I'm just saying this should be a two way street of mutual respect.
On Sat, Mar 11, 2023, 09:11 chakalakasp @.***> wrote:
I mean. I get your frustration. But your main points seem to be that:
1.
The Automatic1111 extension system is cobbled together and lets any extension developer completely jack with the underlying platform in a way that can screw with other extensions’ code. 2.
The Automatic1111 Python library is an insecure dumpster fire. (Which I’d agree with). So you stopped using it in your extension and installed another. Which breaks everything for everyone else but hey, see point 1. You’re right, they’re wrong, if they don’t want their car door dinged they shouldn’t have parked so close.
Which from a point of developer vs developer I understand — the logic is sound. But from the end user perspective, you’ve essentially ended your extension.
The end users mostly seem to be giving your extension a wide berth. Remember; they don’t care about palace intrigue or coder want fights — from their perspective, it worked great at first and then it kept getting “updates” that made users feel unable to reproduce the good results they got from early versions. And then it started getting “updates” that broke core functionality. And then it started getting “updates” that broke the core functionality of all of the other extensions so that now people are resorting to batch commands and power shell scripts to auto update their extensions. You may feel that this is an unfair characterization. The end users don’t care, they just see what they see and move on.
Most of the YouTube tuts out there and reddit threads now point people to Koya_SS and tell people to avoid your extension. Users really don’t have patience for this kind of thing when there are alternatives. I’m guessing the train has left the station on this whole project and you’re not on it. I see you’re now making your own train with blackjack and hookers that completely abandons gradio and A111 and I genuinely hope you are successful.
— Reply to this email directly, view it on GitHub https://github.com/d8ahazard/sd_dreambooth_extension/issues/982#issuecomment-1464931693, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2755J4YVKDYZF4G6KYOMVLW3SIYTANCNFSM6AAAAAAVMKP5DE . You are receiving this because you commented.Message ID: @.***>
I write that as common user and fan of your extension at beginning - why not now? It gonna be surprise but because of community pressure. I had 24GB VRAM card since all that happened, first versions of plugins works best and weren't loaded with all those stuff that people with less VRAM needs - sadly I'm minority here and my right is like truck on road - bigger have less rights. Results become less ideal and as expected? So I stopped use it and swap into Kohya after couple failed results with automatic dream booth. Recently I heard that your extension get few solid fixes so I installed it and...found about above situation. Will uninstall everything and setup fresh automatic1111 without DB and gonna wait to see what you are doing. If you are creating separate software that is independent from automatic? Im all in - just like with kohya. Training is something that is and should be done not in automatic but as separate thing - and popularity of kohya proved its a correct path. Sadly if you update plugin, it might require update from automatic so we end up by clicking update everywhere. Love your work man/woman (who knows :) ) and fingers cross that screenshot will be something more very soon. Ahhh and one more thing, I agree with your old comment about stupid automatic constant updates that break everything and create infinite amount of work to creators. There should be single version and dev version with merge once per month or something - just like are doing normally in software dev cycle (based on Blender for example).
Edit: After 4h of trying recovery from past weeks backups, countless unknown errors and not working clean install, i bring back backup copy created before attempt of clean instal. Well, at least i can generate pictures - im willing to be forced manually update each single plugin if things looks like that. Can't say, Im quite salty atm. and I understand reasons of update that single thing, but instead push such large and breaking stuff thing in main git repo, I wish to see branch of it so users are aware what they are doing. Didn't know how fragile house of card automatic1111 become. Everything shit themself if remove single element and for me as 3D artist forced to google each single thing like "how reinstall python on windows" im mad...mad that I was tempted to re-check dreambooth extension instead stick to kohya.
Yhhhhh. I suggest (if its even possible) to remove DB extension or anything to secure others from same situation.
"@AUTOMATIC1111 bump GitPython to 3.1.30 because some people would be upset about it …"
Yay, at least something
This issue is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days
Kindly read the entire form below and fill it out with the requested information.
Please find the following lines in the console and paste them below. If you do not provide this information, your issue will be automatically closed.
` Python 3.10.9 Commit hash: [0cc0ee1b] Installing requirements for Web UI Skipping dreambooth installation. Dreambooth revision is : Last version Diffusers version is ... Torch version is 1.13.1+cu117 Torch vision version is ...
`
Have you read the Readme? Yes
Have you completely restarted the stable-diffusion-webUI, not just reloaded the UI? Yes
Have you updated Dreambooth to the latest revision? Yes
Have you updated the Stable-Diffusion-WebUI to the latest version? Yes
No, really. Please save us both some trouble and update the SD-WebUI and Extension and restart before posting this. Reply 'OK' Below to acknowledge that you did this. Ok
Describe the bug
This is my first time installing Dreambooth, after installing, noticed I can't update any extension, also I can't select any model from dreambooth "select model" field.
Provide logs
If a crash has occurred, please provide the entire stack trace from the log, including the last few log messages before the crash occurred.
Environment
What OS? Windows 10 64 bit
If Windows - WSL or native? Native
What GPU are you using? Nvidia geforce 2070
Screenshots/Config If the issue is specific to an error while training, please provide a screenshot of training parameters or the db_config.json file from /models/dreambooth/MODELNAME/db_config.json
https://i.imgur.com/gucT2gu.png https://i.imgur.com/XmdkoRD.png