Mega Thread for Non-Technical Discussions

[lllyasviel]

Hi Users,

We have been informed that GitHub has recently deleted several comments and posts due to violations of their Hate Speech and Discrimination Policy.

To foster a healthy environment for project development and ensure that diverse voices can share ideas, all future discussions unrelated to technical aspects of the codes must only take place in this thread.

Please note that non-technical comments posted elsewhere will be moved here or removed by the admin or GitHub if they violate GitHub's Terms of Service.

We value your feedback and are committed to listening to different perspectives to develop better tools and strengthen a healthy relationship between artists and AI techniques.

At the same time, ensure that your comments adhere to GitHub's Terms of Service, particularly this and this and this guidelines.

Thank you for your continued support!

[stellar-eclipse]

Why did you delete all the posts in discussions and then close the discussions area of this repo?

---

Admin Notes

The repo owner did not delete the mentioned discussions. Some discussions disappeared due to deletion by GitHub for TOS violations or by the users themselves. You can still view many previous comments; see the links in the “referenced” block above.

[SleepyDeepy77]

Why did you delete all the posts in discussions and then close the discussions area of this repo?

Probably because the issues were being spammed with off topic garbage and borderline death threats.

[davizca]

Glad to see a bit of coherence after all the hate storm lol. In the case that you guys receive death threats, insults (for being anti or pro AI) or you see insane amount of spam in the place that is not the right one (and that it's continuous), just use the report button. Other than that have a great night (or day!).

[LizThompson93]

@lllyasviel What were you feeling when you downloaded thousands upon thousands of videos of artists who livestreamed or uploaded a video of their painting process? That was the source of your training data.

Pride?

I'd like to know your methodology on how you selected the artists. Did you simply crawl through Youtube, Vimeo, or other video hosting platforms and do a string contain statement for "anime", "Japanese", or did you have a list of artists you specifically targeted and ran through a loop to scrape everything they had?

[LizThompson93]

@Github Admins

Thoughts about your own rule?

"We do not allow content or activity on GitHub that: infringes any proprietary right of any party, including patent, trademark, trade secret, copyright, right of publicity, or other right."

If an artist found that his/her video was taken and used without permission for this model, surely that falls under this umbrella?

[SleepyDeepy77]

If an artist found that his/her video was taken and used without permission for this model, surely that falls under this umbrella?

Presumably it doesn't. They have their own models trained on code and are defending that training generative models is non infringing. Would be weird if they took issue with this project, even if the author did what you claim

[LizThompson93]

I just scanned the group's history and looked at the most recent papers.

1) Sandwich Town with permission, existing art 2) SmartShadow using a mix of real data and synthetic data.

3) Artwork traced by the team of style2paints from an artwork by David Revoy with permission, existing art 4) A small initial dataset of 56 illustration and line drawing pairs was collected by searching for the keyword “illustration with line drawing” on Pixiv (Japanese online artist community)

5) Took a million illustrations from Danbooru artists

6) Human Faces: 10,000 images collected from the FFHQ dataset; 10,000 images collected from various animations, which include works from producers such as Kyoto Animation, P.A. Works, Shinkai Makoto, Hosoda Mamoru, and Miyazaki Hayao.

Some of them were copyright free, some of them were used with permission from the artist, and most of them were extremely questionable like from Pixiv, Danbooru, and established Japanese artists and animation studios. Just from these papers alone they have taken from existing art. Forgive me if you think me being skeptical about this particular process being non-infringing and coming from Videocraft (or branched without lvdm).

[SleepyDeepy77]

Yes, we are all aware of the act of scraping images and training models on them. I'm telling you that it's an open question if doing so is covered under fair use or not. GitHub itself has a product called copilot which was trained on copyright protected code, but argue it was fine for them to do so.

[LizThompson93]

What are your thoughts behind the motivation behind the rapid strategy these tech companies and AI research group are largely employing of asking for forgiveness later instead of asking for permission?

It's a fun thought experiment that if training any material is accepted by the courts, it would be fun to take any copyrighted art, movie, or games with a sample size of 1 and a weighting 1 and pass it through my own model, that would be my very own work now.

[SleepyDeepy77]

Over fitting on the output side would still be an infringement, but those are pretty rare if you dedoup your dataset and do things properly :)

[kknowk6352410]

In the future, you may see robots wielding watercolor brushes and painting in a park, because they have mastered the human steps of painting. Whether this is a fascinating development or a terrifying one, I do not know。

[LizThompson93]

Over fitting on the output side would still be an infringement, but those are pretty rare if you dedoup your dataset and do things properly :)

Oh? Is there a law that states overfitting is illegal? Or that the generated work must be this different? In what way and how much?

[SleepyDeepy77]

Oh? Is there a law that states overfitting is illegal? Or that the generated work must be this different? In what way and how much?

It's just regular copyright law https://en.m.wikipedia.org/wiki/Substantial_similarity

[LizThompson93]

It's just regular copyright law https://en.m.wikipedia.org/wiki/Substantial_similarity

"To win a claim of copyright infringement in civil or criminal court, a plaintiff must show he or she owns a valid copyright, the defendant actually copied the work, and the level of copying amounts to misappropriation."

Except I didn't copy, I used it for "training" purposes.

"Misappropriation is the unauthorized use of another's name, likeness, identity, property, discoveries, inventions, etc. without that person's permission, resulting in harm to that person."

You mean like generative AI?

Companies can't have it both way then.

[cozytraveler]

Some of them were copyright free, some of them were used with permission from the artist, and most of them were extremely questionable like from Pixiv, Danbooru, and established Japanese artists and animation studios. Just from these papers alone they have taken from existing art. Forgive me if you think me being skeptical about this particular process being non-infringing and coming from Videocraft (or branched without lvdm).

It doesn't matter, you don't need permission, court has already ruled that training AI with copyrighted material is legal. It doesn't even copy anything, all it does is learn. He is allowed to take videos from any sources online

https://www.techspot.com/news/103744-developers-suing-over-github-copilot-dealt-major-setback.html

[serick4126]

Some of them were copyright free, some of them were used with permission from the artist, and most of them were extremely questionable like from Pixiv, Danbooru, and established Japanese artists and animation studios. Just from these papers alone they have taken from existing art. Forgive me if you think me being skeptical about this particular process being non-infringing and coming from Videocraft (or branched without lvdm).

Just for your information, the law in the country you referred to as "Japan" permits using a dataset collected without the copyright holder's permission for learning purposes, except when the collector uses the data for their own enjoyment. In this context, "learning" makes no distinction between human learning and machine learning. Of course, this law applies to actions conducted within Japan, so this is merely supplementary information. However, I recommend considering the implications of bringing up data created under such laws in the context of the discussion you wish to have.

[XieBaoshi]

I appreciate new technologies for AI art, but this looks more like a way to deceive someone. What's the point of fake timelapses?

[wenzhenze]

Very cool technical bro, don't mind what others think, do what you want to do.

[stellar-eclipse]

Why did you delete all the posts in discussions and then close the discussions area of this repo?

Admin Notes

The repo owner did not delete the mentioned discussions. Some discussions disappeared due to deletion by GitHub for TOS violations or by the users themselves. You can still view many previous comments; see the links in the “referenced” block above.

You are a shameless LIAR.

You said you had not deleted the so-called mentioned discussions, but I mean the GitHub Discussions area of this repo. You know what it is but you chose not to answer the true question to pretend not to know because you deleted posts in discussions and closed this area like this:

[victorca25]

@stellar-eclipse it was not deleted by him, it was deleted by GitHub because it broke ToS.

You're also now falsely accusing the author here and doing the same the discussion was taken down for. Be rational, you're looking mighty foolish right now.

[stellar-eclipse]

@stellar-eclipse it was not deleted by him, it was deleted by GitHub because it broke ToS.

You're also now falsely accusing the author here and doing the same the discussion was taken down for. Be rational, you're looking mighty foolish right now.

Well, you can say it is the responsibility of Github anyway but yours. But Github won't close the discussion area without closing it manually.

By the way, you are trying to say you are rational by belittling others?

[serick4126]

@stellar-eclipse As you already understand, the reason Discussion #3 was deleted is due to GitHub's actions, as lllyasviel has stated. Upon actual verification, JapanYoshi's user page (https://github.com/JapanYoshi) no longer exists. And while he hasn't specifically addressed why the discussion area was removed, the fact that you can post in this ISSUE #41 indicates that the discussion area has simply been relocated and still exists. Therefore, all your reasons for insulting him by calling him a "shameless LIAR" have been invalidated. Now is the time for you to apologise. Everyone makes mistakes, but it is always possible to acknowledge and apologise for him.

[glektarssza]

@serick4126 @lllyasviel If you can provide some kind of proof that GitHub staff are able to or did disable discussions on this repository then I might be inclined to believe this explanation. I have never heard of GitHub staff being able to do this before. At most I've heard of them deleting accounts or disabling entire repositories and, as a result, those accounts/repositories being replaced by a "ghost" (as seen in issue #7). The only other situation in which I've seen discussions get disabled is by the repository/organization owner. In this situation, if that is what happened, it would not paint the people behind this project in a particularly positive light.

This is not an accusation, mind. I am not opposed to generative AI in principle when used appropriately. I just find it very odd that there are claims being made that the GitHub staff did something I have never heard of them being able to do in the past, ever and would like clarity/evidence of this particular claim as, if it is true, it's somewhat concerning.

[davefojtik]

I am sorry @lllyasviel, but this is too far over the line we shouldn't have crossed. I am very well aware of the already existing problem on social media where scammers try to impose themselves as artists to make quick money. This tool will benefit just and only them. This really won't help calm the already turbulent waters between artists and AI developers.

Since I am not only a RunPod implementation maintainer of your great project Fooocus and contributor to the Fooocus-API but also an active member of art communities, I can't afford to be associated with tools that are aimed at such activities, therefore pausing my work on these projects until this controversy is resolved. I am sad this had to happen and I wish you good luck.

[AndreiSva]

I am sorry @lllyasviel, but this is too far over the line we shouldn't have crossed. I am very well aware of the already existing problem on social media where scammers try to impose themselves as artists to make quick money. This tool will benefit just and only them. This really won't help calm the already turbulent waters between artists and AI developers.

Since I am not only a RunPod implementation maintainer of your great project Fooocus and contributor to the Fooocus-API but also an active member of art communities, I can't afford to be associated with tools that are aimed at such activities, therefore pausing my work on these projects until this controversy is resolved. I am sad this had to happen and I wish you good luck.

Yes, we should.

[dogtopus]

It's a fun thought experiment that if training any material is accepted by the courts, it would be fun to take any copyrighted art, movie, or games with a sample size of 1 and a weighting 1 and pass it through my own model, that would be my very own work now.

Copyright law doesn't work that way. By doing this you are demonstrably copying expressions since it's the same work in and the same work out. When we talk about the fact that AI training on copyrighted work is not infringement, there's always this prerequisite of training your models properly, since the amount of expressions copied by the model is, in a high level view, a slider that the model developer is able to control. Sliding it to 100% and say "oh I didn't copy anything because AI training is legal" is likely not a valid claim in the eyes of law.

BTW this is also the same technicality that allows "clone" or "compatible" software to exist, without infringing the copyright of the originals. One develop the clone based on the behavior of the original, but never copy any code directly, therefore they still "used" the original in production but not demonstrably copying anything as far as copyright is concerned, therefore it's completely fine. (Of course, unless there are other obstacles like algorithm patents.) But once code is copied and it crosses some plausible threshold (like a significant chunk of code was copied verbatim, not just some public domain things like quick sort), it crosses the line.

[serick4126]

@glektarssza By mentioning me, are you asking for my input? It seems your claim is that it was lllyasviel, not GitHub, who closed the discussion area. I agree with you on that point. Of course, even if you ask me, I can't provide any evidence or reasons to convince you, and I'm okay with that.

TBH, I was subscribed to Discussion #3, so almost all the posts are saved in my Gmail. Just looking at the awful content, I can kind of understand why he decided to close the discussion area.

What concerns me is the act of slandering someone without grounds. Fortunately, I was born and educated in a country where I am not restricted from holding any ideology, so I see no issue with your doubts. And I have no intention of changing your opinion.

[glektarssza]

@serick4126 Your mention of discussion topic 3 seemed to suggest that it was deleted due to the actions of GitHub staff. To quote you directly:

As you already understand, the reason Discussion #3 was deleted is due to GitHub's actions,

Reading back at my post I can see that it was not clear that I was mentioning you because I was wondering if you had any proof that the deletion of topic 3 was done by a GitHub staff member. My apologies there. I was not asking if you had proof that the discussion area was closed by @lllyasviel as there is no reason to think you would have any kind of evidence on that.

Let me clear up what I really was trying to say in my original comment.

• I was asking if you had any proof that the particular topic mentioned (topic 3) was deleted by a GitHub staff member.
• I was simultaneously asking @lllyasviel if they had closed the discussion area themselves, if they did it at the request of a GitHub staff member, or if it was done for them by a GitHub staff member.
• I was ask that, in the event either of the above was done by a GitHub staff member, if there was any evidence to support those claims.

The reason I was asking was mainly to try to help foster transparency and an open dialog. Having the discussion area suddenly vanish is not a good look for the project owners when there are other moderation tools available that might have been able to help deal with the apparent deluge of hatred that was going on (which I did not witness, to be clear).

Reading back on this entire thread I should also admit that I did misread along the way and interpreted another users comment around the disappearance of the discussion area as an implication that @lllyasviel removed it on purpose to censor discussion and that biased my original comment. You have my apologies for that, @lllyasviel, as that was not my intention.

[serick4126]

@glektarssza I understand why I caused confusion. My statement, "As you already understand, the reason Discussion #3 was deleted is due to GitHub's actions, as lllyasviel has stated," was in reference to stellar-eclipse's comment, "Well, you can say it is the responsibility of GitHub anyway but yours," which was right above mine.

In response to stellar-eclipse's point, "But GitHub won't close the discussion area without closing it manually," I explained that the "discussion area" has merely been relocated and still exists here. While I omitted this before, I also believe that it was lllyasviel who closed the discussion area as a function of GitHub's platform.

Regarding the initial deletion of Discussion #3, I proceeded on the premise that stellar-eclipse already recognised it as GitHub's action. As you rightly pointed out, I lack concrete evidence for that claim. I apologise for any confusion this may have caused.

[IMTB682]

Let me clear up what I really was trying to say in my original comment.

• I was asking if you had any proof that the particular topic mentioned (topic 3) was deleted by a GitHub staff member.
• I was simultaneously asking @lllyasviel if they had closed the discussion area themselves, if they did it at the request of a GitHub staff member, or if it was done for them by a GitHub staff member.
• I was ask that, in the event either of the above was done by a GitHub staff member, if there was any evidence to support those claims.

@glektarssza Maybe it's because I reported the user to Github.

[glektarssza]

@serick4126 Thank you for clearing that up.

@IMTB682 It's possible but I personally think it's unlikely that reporting a single user would cause an entire discussion topic to get deleted. Usually when a user gets deleted (whether due to them being banned or some other reason) they are replaced by a "ghost" user (see issue #7 for an example of this).

[IMTB682]

@glektarssza I'm not sure, but if you search for the user on github, you will find that all his/her replies and posts have disappeared. Since he/she is the original poster of that thread, it's not surprising that the thread is gone. However, I see that the user is still giving dislikes to others lol, so it might just be a temporary suspension.

[Debuggernaut]

So many so-called "artists" in here are suffering from skill issues

Any jerk can learn to draw in a couple of months, at least well enough to look at a piece of AI art and replicate it over the course of painstaking hours of sketching

The thing that makes it actual art is the artist expressing something, and an AI bro or a little kid with a big idea is just as much of an artist as someone who somehow needed years to learn to draw and paint.

BTW: Even if copyright law was somehow found to be infringed, someone would cook up a new dataset that works around it and re-train all these models. So you'd better start figuring out where you're going to fit into the new world, because it's coming one way or another.

[packetlost]

What are your thoughts behind the motivation behind the rapid strategy these tech companies and AI research group are largely employing of asking for forgiveness later instead of asking for permission?

It's a fun thought experiment that if training any material is accepted by the courts, it would be fun to take any copyrighted art, movie, or games with a sample size of 1 and a weighting 1 and pass it through my own model, that would be my very own work now.

You demand others live on their knees begging for approvals. That is how you may live, since it suits you, everyone else may disavow Ludditism and not harbor some kind of perverse love of copyright. #KopimismForever ARRR matey! Data should be free! No gatekeepers. No masters.

[packetlost]

So many so-called "artists" in here are suffering from skill issues

Any jerk can learn to draw in a couple of months, at least well enough to look at a piece of AI art and replicate it over the course of painstaking hours of sketching

The thing that makes it actual art is the artist expressing something, and an AI bro or a little kid with a big idea is just as much of an artist as someone who somehow needed years to learn to draw and paint.

BTW: Even if copyright law was somehow found to be infringed, someone would cook up a new dataset that works around it and re-train all these models. So you'd better start figuring out where you're going to fit into the new world, because it's coming one way or another.

Thaler v. Perlmutter birthed a golden age without the abomination of copyright. AI art has seized the means of production for the people. Those that could not before, now can and that is called democratization. Only the rentseeking landlords of the art world weep for the death of the dark age.

[cavecomics]

Re: " Users must not employ the tool to generate false information or incite confrontation." We still haven't heard what the legitimate use case scenario is..... What is the legitimate use case scenario?

It cannot possibly be for "education" since there are millions of YouTube videos already doing just that and showing a step-by-step process with actual voiceover so you know the MEANING behind each step being taken. Because seeing a line drawn doesn't teach anything without knowing the purpose behind that line.

Is it for entertainment value then? But, again, you can just go YT/TT/IG and watch millions of legitimate timelapses there... There is no shortage of this entertainment and many artists try to make them more interesting to watch since it helps them promote their art.

I don't see any purpose for this other than to deceive. What is the assurance that users WON'T use this for deceptive purposes? There is none. It's far more likely that someone will use this as a means to generate false information rather than for any ineffective educational/entertainment value.

I can already see the down votes here, which are already being made on many legitimate comments that have gone unanswered, but I think it's critical that this inherent issue be addressed already and I am doing so as respectfully and neutrally as I can. I believe that people who do want this project to succeed would WANT this to be addressed right away and succinctly so it can move forward without other people getting upset over it and put the conversation to rest.

The only reason I can see why this hasn't been addressed is because the creators know there isn't a legitimate use case and the purpose is to deceive but are not willing to admit that. So, please, prove me wrong.

[packetlost]

We still haven't heard It cannot possibly be for Is it for entertainment value then? I don't see any purpose The only reason I can see

Look at all of your lack of imagination; over and over. What YOU are unable to think of is not a limiter for anyone but you.

[cavecomics]

We still haven't heard It cannot possibly be for Is it for entertainment value then? I don't see any purpose The only reason I can see

Look at all of your lack of imagination; over and over. What YOU are unable to think of is not a limiter for anyone but you.

Okay, so then using your superior imagination, what is the legitimate use case?

[packetlost]

Everything that isn't illegal is legitimate. In your bigotry and seething hatred, your closed mind allows you to see nothing.

Weep for the property owners in your Ludditism. You can cosplay the Dey-terk-er-jerbs meme if you like... such is freedom. But, remember, if AI can take your job, you didn't have a skill. It is the same with the anti-immigration people, if an undocumented worker can walk a thousand miles and take your job, you deserve to have your job taken.

GetGudNotButthurt

[zinongli]

A keyboard allows one to type their idea, I don't see an author's job taken by "typist bros". If you are actually creating art, a tool/platform wouldn't be able to compete with you. Unless, you excel in your field not by ideas and creativity, but by skills only. In the latter case, you aren't any more proficient than AI or a printer.

[davefojtik]

For anyone coming outside of GitHub and the dev world, I just want to add a note... This repo and discussion is something like a Reddit bubble. It attracts very specific types of people and it does not represent casual programmer behaviour. We are capable of being quite normal human beings and other GitHub projects are usually professional, respectful and friendly environments. Also, I wouldn't recommend the art community joining such discussions in the first place as the AI fanbase is very opinionated.

[cavecomics]

For anyone coming outside of GitHub and the dev world, I just want to add a note... This repo and discussion is something like a Reddit bubble. It attracts very specific types of people and it does not represent casual programmer behaviour. We are capable of being quite normal human beings and other GitHub projects are usually professional, respectful and friendly environments. Also, I wouldn't recommend the art community joining such discussions in the first place as the AI fanbase is very opinionated.

You seem to be assuming a lot of things here... I am not a programmer, but as a graphic designer I do have to have some basic programming knowledge and I do occasionally use GitHub, albeit not as much as others. I am very much so open to the tech industry, I have worked in it before and would like to work in it again.

I don't think me being part of the art community - which is directly affected by these projects - is reason to exclude me and others from the discussion. I think I am being VERY neutral in simply asking what exactly the legitimate use case scenario is for this project exactly especially when the devs themselves have pointed out that it should NOT be used for generating false information (which I assume to mean, impersonating that the AI generated image was made in a program other than an AI image generator).

Note that I do not use inflammatory language such as "AI bro" - which I vehemently hate - or, uh, "Luddites"... I have one simple question, still unanswered.

[packetlost]

I don't think me being part of the art community - which is directly affected by these projects - is reason to exclude me and others from the discussion.

First, you are wearing a dey-terk-er-jerbs costume. Second, they don't let the party that believe they are aggrieved be the judge or be on the jury now do they? There is the overt conflict of interest and an obvious presumption of partiality. A "neutral tone" does not make one neutral.

[packetlost]

Also, I wouldn't recommend the art community joining such discussions in the first place as the AI fanbase is very opinionated.

Unlike the "art community" that is entirely devoid of opinions on the matter. Dey-terk-er-jerbs hysteria is pure unadulterated fear. The fearful are notoriously unreliable thinkers... Embrace technology or be steamrolled by it.

[glektarssza]

Just chiming back in here (probably for the last time); the reason I originally dropped my hat into this conversation was over concerns about a lack of transparency about what actions were being undertaken by GitHub staff versus the owners of this project. I still, to this point in time, have not gotten a clear answer. Sometimes silence speaks louder than words, at least in my personal opinion. Particularly now that we're seeing, in this very thread, what it looks like when GitHub staff do step in to take actions. See some of the comments by @packetlost that have been marked in violation of GitHub's Acceptable Use Policy above and issue #7 which was created by a "ghost" user (which is a user whose account no longer exists for some reason).

I agree with @davefojtik; this repository and this issue thread in particular are not representative of most of the communities the people involved come from, developers or otherwise. I know it doesn't represent very well what I'm usually like in my day to day life and interactions online. Hence why I'm muting this repository and won't be participating any further in this discussion. I considering the silence from the project owners a sufficient "answer" to the concerns I raised.

I hope you all have a wonderful rest of your weekends, wherever you are in the world. ❤️

[packetlost]

what it looks like when GitHub staff do step in to take actions. See some of the comments by @packetlost that have been marked in violation of GitHub's Acceptable Use Policy

Oh, I am sure that these overly-emotional people had an overly-emotional reaction to the truth. They want what cannot be. AI cannot be stopped. AI cannot be contained. Nobody anywhere should be so very short-sighted as to fight progress. All who have ever done so have been defeated by it. For what reason can one justify one's position of choosing to be losing? Perpetual, guaranteed defeat is not strategy nor policy. Systemic defeatism produces no gains, only loses.

So, yes, my 3090 go brrrrt minting images, oOoOoh how I admit my "flaws." But, their faces brrrrt only tears and gibberish. Fear and illogical cries of dey took er jerbs only reinforces what everyone else can clearly see, Ludditism is a meme that is doomed from moment one. What they really find "unacceptable" is that no amount of tears change truth and no amount of censorship can make that go away. That which cannot be... will never be.

[randucampos]

can you make one click installer please?

[noxiouscardiumdimidium]

Potential Synthetic Data Generation Workflow:

Original Dataset becomes training regularization images. Batch process OD images, and output intermediary frames instead of video. (omitting first and last key-frames) Auto crop into tiles to selected resolution , and tag each tile with WD 1.4, and a captioner that only captions color/contrast/style/render-style/quality keywords. automagically create proper Koyha_SS folder and naming structure auto populate basic Koyha_SS training .json config with basic best practice settings for full model and lora training

[imljc]

Technology created to deceive others has no value. all I have see the word "Profiteering" in it.What a shame,what a crock of sxxt.

[imljc]

do you believe what you saying? "strengthen a healthy relationship between artists and AI techniques."

What a clown,what a shame of you,what a crock of sxxt.