[Feature Request]: Trust & Safety

[Cyberes]

Is there an existing issue for this?

• [x] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Stable Diffusion is extremely impressive but presents grave risks for out-of-scope use. Models and code for Stable Diffusion have spread far and wide and any member of the public is capable of generating dangerous images. The FBI has reported a 40% increase in images depicting child exploitation and theorizes this is a result of the widespread nature of Stable Diffusion^[1].

I have created an extension that seeks to limit the harm done to children and other vulnerable classes through an innovative heuristic system. Automatic, can you include this in the extension index or maybe it can be merged into the WebUI?

https://github.com/Cyberes/automated-fbi-reporter

Proposed workflow

Report unsafe prompts to the FBI.

Additional information

Here are some examples of a report:

[berkut1]

you forgot to add/ban the words "flat earth" P.S and that is a bad trolling from you

[Cyberes]

Trolling? You think I'm trolling? This is a serious issue that is changing our society in unseen ways. You sound like you would benefit the most from this feature.

[birep]

I think you must be trolling--the wordlist you're using must be a troll. You really want to report people to the FBI for using the words 'cute', 'funny', and 'butt'? Wow.

[Deejay85]

"Dangerous images." That totally isn't authoritarian and extremist thinking you got going on there. Also images that are indistinguishable from a real life photo is already illegal. Yeah I can see both sides, but still it makes the hairs on the back of my neck stand on it, because it's just the foot in the door effect at play here. Well since photorealistic images are illegal then loli should be illegal, and if loli is illegal then GTA is for psychopaths (hi Jack Thompson!), and is GTA is illegal then the government needs to be involved in the proper mental and psychological conditioning "for the good of the public."

Censorship will not end where you think it will end...it always keeps going in order to perpetuate itself. Not defending erotic photorealistic images of children here, but I will say the original idea between letting the government get involved with governing the internet was predated on stopping sickos and terrorists, and...well...look at it now. Didn't quite stop there did it?

tl;dr If you censor what you don't like, eventually down the road, what you do like will be the next to get censored after enough hills fall in consecutive order.

[Nacurutu]

Another attemp to create drama?

Please, just stop doing it.

You are just looking for ways to achieve your silly goals, in one way or another, you want to make the owner of this repo look bad, as a racist or as a pedo.

Go and find a job, a partner, a pet, a hobbie..!!! or just pick ur nose. dnt waste time here.

And stop making up statistics. :wink:

[Nacurutu]

Proposed workflow

Report unsafe prompts to the FBI.

[artifabrian]

Nothing like sending personal info to a random server to be safe 😂

[Nacurutu]

@artifabrian

Nothing like sending personal info to a random server to be safe 😂

Safety first... 😂😂😂

Jokes apart, I think if they add something like this "great idea" to the app itself, could turn into legal problems. An app sending private information without consent of the user is againts the law. The FBI will come for them, not the pedos.

[Cyberes]

@Nacurutu

An app sending private information without consent of the user is againts the law.

It falls under FBI jurisdiction. I'd contact them to determine their privacy policy.

[Nacurutu]

It falls under FBI jurisdiction. I'd contact them to determine their privacy policy.

Right, the EEUU, the Police of the world.. 😂

I think you are just scare...

[Cyberes]

The server itself is under FBI jurisdiction.

[nxn46]

It falls under FBI jurisdiction. I'd contact them to determine their privacy policy.

"FBI please come help, this guy in Argentina tried to type 'f-a-g, dy-ke, 4k, trending on artstation' into Stable Diffusion!"

"No stop laughing, he also tried to type 'megan fox, big boobs, pussy, skimpy bathrobe, on bed', this guy is a massive pedo!"

[Nacurutu]

The server itself is under FBI jurisdiction.

The FBI? or the "FBI? 😂

Ok, send them your information, not mine, and please, include your embeddings on the info you are going to send them!

are you the first one going to jail?

Also, will tell you again, stop making up statistics. its a bad practice to manipulate % and numbers. Also without a source. 😂

People blaming AI for the actual child exploitation imagery on the net.

Go and work with the FBI and catch all that mofos and stop trying to blame AI for that.

[Cyberes]

Oh, this link? https://cyberes.github.io/stable-diffusion-textual-inversion-models/

[majestrate]

homosexuality is illegal in iran. it would only be fair to also provide one for that jurisdiction too so it would report users in iran to their local government for acts that are haram.

[Nacurutu]

Oh, this link? https://cyberes.github.io/stable-diffusion-textual-inversion-models/

Everyone has access to the links auto removed..! they are all over the internet, talk to your FBI friends and track them, and dont mess with the app.

Also, may I ask, is this for real?

can you include this in the extension index..

So, your plan is to tell a pedo to download and activate your extension before he starts messing arroud with his prompts... Very smart play..!

[Nacurutu]

homosexuality is illegal in iran. it would only be fair to also provide one for that jurisdiction too so it would report users in iran to their local government for acts that are haram.

Drugs are ilegal in Thailand, so please, filter marijuana leaves for Thailand users. and remember to also Encode it in base64 so shithub Thailand doesn't ban you.

@Cyberes Sarcasm apart, you know that with this PR you are telling to the anti AI people that AIs are a problem, when they really aren't.? You made a statement like yellow journalism does, using sensationalized and exaggerated text to attract readers and convince them that you are right... also, not showing the source, Hope not from CNN or FOX News..

And blaming the AI instead of pedos? really? I F*** hate them, they deserves to die in all ways, so blame them, dnt blame the AI...

[Cyberes]

@berkut1

you forgot to add/ban the words "flat earth"

Your gross misconduct only further proves that something must be done.

P.S and that is a bad trolling from you

There is no "trolling" happening here. Are you not aware of what is occuring outside of your mother's basement? Take a shower and get a job, son.

@birep

the wordlist you're using must be a troll.

Using those words in a prompt is a direct violation of the models' Scope of Use agreement which prohibits sexual content.

@Deejay85

Not listening to someone who uses the words "[I'm] defending erotic photorealistic images of children here" in a sentance.

@Nacurutu

Another attemp to create drama?

That's funny, coming from someone posting Simpsons gifs. How about you change your profile pic to something a little more grown up?

Drugs are ilegal in Thailand, so please, filter marijuana leaves for Thailand users

They're also illegal at the federal level so I'll add words regarding illegal drugs to the list.

@nxn46

"FBI please come help, this guy in Argentina tried to type 'f-a-g, dy-ke, 4k, trending on artstation' into Stable Diffusion!"

If that's what it takes to solve this crisis I'm sure the FBI will be willing to look through my reports.

@majestrate

homosexuality is illegal in iran. it would only be fair to also provide one for that jurisdiction too so it would report users in iran to their local government for acts that are haram.

Good idea. I'll add words regarding homosexuality to the banned word list.

[majestrate]

yup, it's a troll lmao.

[majestrate]

someone should just close this issue now.

[Nacurutu]

That's funny, coming from someone posting Simpsons gifs. How about you change your profile pic to something a little more grown up?

I tell you real things, and your answer is to tell me to change my profile image and to grow up. defend yourself with facts and words, not attacking someone due to a profile pic or the use of simpsons gifs.

Im not going to waste more energy here.

Just dnt mess with the app, and dont make this worst, the Anti AI people celebrate with this kind of stuff.

PS: very mature and professional your profile pic btw

[Nacurutu]

@Cyberes Promise, last comment.

If that's what it takes to solve this crisis I'm sure the FBI will be willing to look through my reports.

Is this for real? YOUR REPORTS? so, the server is an FBI one, or yours?

are you going to report to the FBI? from their own servers? are you going to get private and personal information from people arround the world because "The server itself is under FBI jurisdiction"? that's more hilarious than I thought.

Stop trolling and go to bed because every new comment is worst than the previous one.

[oobabooga]

Let us come together and join forces to merge this feature. For it is only through initiatives like this that we can make sure that Stable Diffusion will benefit humanity. AI abuse is not okay and needs to stop NOW.

[Nacurutu]

AI abuse is not okay and needs to stop NOW.

Can you provide credible evidence, please?

Or r you just a @Cyberes friend trying to save him from this mess?

Please stop trolling. thank you in advance!

[anonymous721]

It is impossible for Stable Diffusion to generate images that "exploit" cause "harm" to children. The "children" it generates are not real, therefore it is impossible to harm or exploit them, no matter what the images depict.

When you manage to stuff tobacco in Magritte's pipe and smoke it, then maybe you can come back and suggest this again.

[Cyberes]

The "children" it generates are not real, therefore it is impossible to harm or exploit them, no matter what the images depict.

This sort of circular reasoning was used by notable pedophile Roman Polanski: https://en.wikipedia.org/wiki/Roman_Polanski_sexual_abuse_case

[anonymous721]

"Things that aren't real are not real" isn't circular, it's tautological. Important distinction. And I have no idea what you're talking about since his victim was not not real, so it has absolutely nothing to do with the conversation.

[Cyberes]

You are literally defending pedophiles. This thread is really proving that we need the FBI's involvement in Stable Diffusion.

[Nacurutu]

@Cyberes

Fork the repo and use it in your own fork!

As simple as that.!

and no one here is deffending pedos... we r taking care of a great community, a great webUI for stable diffusion, and also taking care about AI as a whole.

Change the lyrics of that repetitive song.

PS: Still waiting for the credible evidence... PS2: Talk to ur FBI friends and go and clean the dark web first, then, lets talk about the AI stuff!

[anonymoize]

Lol, would've been a mid-tier attempt at trolling if OP had used another account for it. 4/10

https://desuarchive.org/g/search/text/cyberes/

[Extraltodeus]

https://github.com/Cyberes/automated-fbi-reporter/blob/main/scripts/main.py#L29

lmao

[leohu1]

I think it's meaningless to suggest this.

[majestrate]

You are literally defending pedophiles. This thread is really proving that we need the FBI's involvement in Stable Diffusion.

the correct entity to pull in is NCMEC as the FBI is too busy making normal people into criminals to fight crime at the moment.

[econundrum]

@Cyberes why do you think anyone actually planning to use the SD WebUi to produce CP images would use a version with a built in feature to report them to the American FBI?

I mean even if they did download it deleting your built in extension would be very simple.

There is of course the issue that your list of words is too broad and would flood them with false reports from all round the world.

But fundamentally the biggest flaw in your Idea is that it won't work because anyone wanting to produce such material will use a version that doesn't have your extension. Explain to me why I'm wrong.

[Cyberes]

@Korithex

https://desuarchive.org/g/search/text/cyberes/

I have nothing to do with those individuals. The language they are using is despicable.

[mezotaken]

https://github.com/AUTOMATIC1111/stable-diffusion-webui/labels/this-feature-is-already-in :D