PetrochukM
commented
4 years ago @erogol What do you think? This should likely be considered before someone gets hurt :(
Closed PetrochukM closed 4 years ago
PetrochukM
commented
4 years ago @erogol What do you think? This should likely be considered before someone gets hurt :(
kdavis-mozilla
commented
4 years ago @PetrochukM What do you suggest to do?
PetrochukM
commented
4 years ago Thanks for taking the time to respond, this is a tricky issue that we need to work through together as a community. As the leading TTS code base, I think there is a great opprotunity to start defining what ethical open source lifelike TTS looks like! :)
Goal(s)
Suggestion Open source an analysis of the project ethics as a markdown or blog post. The analysis should cover the ethical implications of open source life-life TTS. Then, the analysis should be tied back to concrete development practices. Lastly, in order to validate the analysis, there should be some feedback from an expert in AI ethics.
Example Outcome ~A bad actor may want easy access to lifelike TTS to exploit senior citizens with automated phone calls. In order to mitigate that risk, Mozilla doesn't make trained TTS models accessible without going through a basic application system. Furthermore, Mozilla restricts the models from being used commercially.~
PetrochukM
commented
4 years ago Lastly, I'd consider adopting this oath: https://techcrunch.com/2018/03/14/a-hippocratic-oath-for-artificial-intelligence-practitioners/
Some of the principles:
My AI will prevent harm whenever it can, for prevention is preferable to cure.
I will apply, for the benefit of the humanity, all measures required, avoiding those twin traps of over-optimism and uniformed pessimism.
I will remember that I remain a member of society, with special obligations to all my fellow human beings.
nmstoker
commented
4 years ago I appreciate the intention @PetrochukM but these misuses seem hard to defend against from within this repo or based on tying it to analysis, which the skeptic in me thinks may just slow down progress without discernible benefit. I think that the ethical discussion, important as it is, would be more suited to happen at another level and in a more general sense that is detached from this specific project (since the concerns are common to all TTS). Saying this, I'm merely giving personal opinions here and have no formal ethics experience.
If we take your two suggestions of making people go through an application process and restrictions commercial use, these would likely not deter someone motivated by the exploration of senior citizens. Someone who might proceed with that kind of scam will not be concerned about breaking commercial terms because it pales in comparison to the punishment relating to a scam. It would likely be hard to prove in any case. And only an application that severely impacted legitimate users would have the sort of teeth required to stop or slow down a bad actor.
That said, I'll try to keep an open mind and look forward to hearing what others have to say on this.
erogol
commented
4 years ago Pure personal comments. I think it is important to think about ethics of AI projects. However, placing a document about it is kind of futile. Since anyone with malice in mind, would not really read and care about it since the code is already there.
Having user information before the code also hinders the progress. That would keep people using it away and issue some privacy concerns. We cant also really say anyone with the given info is secure to use TTS since it is really hard to verify the real identity.
Being all said, I agree that we need a generic framework against malicious AI usage everywhere not only TTS.
JRMeyer
commented
4 years ago Documentation is futile to combat the evil-doers, but it may convince non-evil researchers be more thoughtful with the models they create / release. Something to the effect of Don't use this code to do bad things is easy enough to write into the CODE_OF_CONDUCT, right?
Not within the scope of this repo, but the only way to fight against malicious use of TTS is to develop technology which can discriminate synthetic speech from real speech, and have that technology embedded everywhere. It's the same issue with GPT-2.
erogol
commented
4 years ago @JRMeyer yeah we can write, but I don't think it will fand off people anyways. But it'd look good :)
erogol
commented
4 years ago PetrochukM
commented
4 years ago Can you please reopen this? I believe there is more that should be done with regard to this issue.
Here is how I came to my conclusion: Context: The current thread has taken the first solution presented as the final solution. Problem: There is a significant risk for harm to other people; therefore, there should be more time, more exploration, and more input taken on this issue. Potential Solution: There should be a more thoughtful approach. For example, we could follow through on the points made in this graphic below.
Good decision chain by the Stanford Decisions and Ethics Center
erogol
commented
4 years ago @PetrochukM you'r welcome to move it to https://discourse.mozilla.org/.
PetrochukM
commented
4 years ago Okay. Thanks for pointing me there. Do you think that forum will lead to a meaningful discussion of this serious issue as suggested in https://github.com/mozilla/TTS/issues/295#issuecomment-547660462?
With regards to issue ownership: Personally, I do not have enough time to take on another ethics project and additional open-source projects; therefore, I need someone's help in taking ownership of this issue. (On some days, I am already working way past 12 hours a day.)
@erogol Do you have time to take on this issue? Would this be a priority for you? A proper ethical process involves careful consideration and attention.
If not, why do you not see this as a priority?
PetrochukM
commented
4 years ago It sounds like this is not a priority issue... which is unfortunate given the significant negative impacts this open-source library could have.
I'd encourage everyone on this thread to read this: https://twitter.com/metaviv/status/1205212076035080192?s=20
The article referenced in the tweet sums up the dominant viewpoint in this thread...
Please consider doing this, at the very least:
m-toman
commented
4 years ago I wonder how one could enforce the technical solutions in an open source project. You can't log what people do, the watermark code can just be removed, agreements ignored.
nmstoker
commented
4 years ago @PetrochukM I note you didn't raise this on the discussion board: https://discourse.mozilla.org/c/tts
As you appear to be involved with a commercial speech solution, maybe you could share some details on the discussion board regarding the AI Ethics approaches your company is considering?
Are you looking at updating your T&C and/or to implement any aspects of the points you raise and how do you see these working in an open source solution?
Do you see any conflict with a commercial software operator suggesting open source projects apply restrictions?
PetrochukM
commented
4 years ago @m-toman There are some initial first steps, for example:
I also don't want to be the only person thinking of solutions. If possible, I'd like some help...
@nmstoker Thanks for digging into my job history... I'm not prepared to be interrogated about our small startup's policies.
PetrochukM
commented
4 years ago My AI will prevent harm whenever it can, for prevention is preferable to cure.
A Hippocratic Oath for AI Practitioners
At the end of the day, I don't expect the people here to solve the world's problems. I expect that we'll execute on basic prevention measures of the misuse of this technology.
nmstoker
commented
4 years ago My apologies, wasn't meant to come across as interrogation, i thought you'd be willing to bring relevant experiences to the discussion but I appreciate if you prefer not to. Btw: no digging required, it's on your profile here.
reuben
commented
4 years ago @PetrochukM I think you are severely underestimating the complexity of this problem. Your suggestions proposed here:
In addition, they would completely derail the focus of this project, which is to implement a high quality open source text-to-speech engine, into solving the general problem of ethical licensing of software and machine learning models.
This discussion is better suited for the Open Source Initiative forums, or the Electronic Frontier Foundation forums, or the Mozilla Governance forum. Not the TTS issue tracker.
PetrochukM
commented
4 years ago The changes I proposed above are simple and don't require lawyers...
@reuben I am actually doing something about this issue by bringing it up and I am leading ethics initiatives in my own company. It's easy to be a naysayer. It's hard to come up with solutions!
reuben
commented
4 years ago Ask for consent from a voice actor from cloning their voice. A simple "Yes" via an email exchange is more than enough.
This is not a solution to anything. We already have consent from voice actors through contracts and licenses. It doesn't change anything for someone using this code on their own, as we don't have control over that due to the MPL license of this code.
- Having a simple vetting process before a user can get ahold of a pre-trained model. For example, the vetting process could be like... "Please email erogol for a Google Drive link to the pre-trained model with your Linkedin and Facebook".
This also doesn't fix anything, due to the MPL license of the pre-trained models. Anyone who gets somehow "vetted" (by checking their LinkedIn? wtf) is free to then redistribute the model without any vetting themselves.
PetrochukM
commented
4 years ago This is not a solution to anything. We already have consent from voice actors through contracts and licenses. It doesn't change anything for someone using this code on their own, as we don't have control over that due to the MPL license of this code.
Actually? Sorry, I looked for an hour for the contract or agreement with Linda Johnson (https://keithito.com/LJ-Speech-Dataset/). I'd love to see it!
I stopped working with Linda Johnson dataset because I couldn't find any evidence that she/he provided consent for her likeness to be used. It's unfair to use her voice if she didn't consent to it.
This also doesn't fix anything, due to the MPL license of the pre-trained models. Anyone who gets somehow "vetted" (by checking their LinkedIn? wtf) is free to then redistribute the model without any vetting themselves.
This language sounds devoid of nuance saying things like "anything" and "wtf"...
I'd love your help with coming up with solutions. It's important to remember that deep fakes have already caused a lot of pain and anguish in our society.
reuben
commented
4 years ago Actually? Sorry, I looked for an hour for the contract or agreement with Linda Johnson (https://keithito.com/LJ-Speech-Dataset/). I'd love to see it!
It is the first sentence after the title, as well as the last sentence of the second paragraph. Linda Johnson licensed their voice recordings into the public domain when contributing to Librivox.
This language sounds devoid of nuance saying things like "anything" and "wtf"...
It is devoid of nuance. I don't think this issue will accomplish anything beyond wasting project resources. It is already closed. I have given examples of places where this can be discussed with the appropriate nuance (legal/licensing/governance forums), and where it actually has a chance of going anywhere. Please stop commenting here as if this complex issue is simply a matter of adopting some bullet point list of ideas. Specially after the issue has already been closed.
PetrochukM
commented
4 years ago Licensing voice recordings is different than licensing your likeness... Putting an audio book in the public domain is far from giving permission for everyone to clone your voice / identity.
Can you help me navigate the various legal/licensing/governance forums? If you think that it'll help make meaningful progress in preventing misuse of deep fakes, I am all about it.
I am unfamiliar with how those forums would stop from the misuse of open source TTS.
it actually has a chance of going anywhere
We can make simple changes now to prevent misuse and you are communicating that there is no point. Like I said before...
It's important to remember that deep fakes have already caused a lot of pain and anguish in our society.
Prevention is preferable to cure
reuben
commented
4 years ago Can you help me navigate the various legal/licensing/governance forums?
I would start by creating a thread on the Mozilla Governance mailing list about how to deal with ethical restrictions in MPL licensed projects. First you probably should read up on software licensing. As I've said here multiple times, you are completely ignoring the licensing issue which is at the core of this problem.
You can describe the problem of ML abuse/deep fakes, and how Open Source licenses do not permit things like usage restrictions or user vetting. You'll then hopefully get some feedback from people who know about software licensing and can help set guidelines for existing and future Mozilla projects.
As for the technical countermeasures like fingerprinting, feel free to make PRs against this repo. I'll gladly review them.
Hey! Thanks for your contribution.
I'm concerned this codebase could be used in a number of bad ways:
Before this effort continues, there should be some ethical consideration that this work won't be used to harm another person, undermine government systems, and undermine the legal system.
The implications are large; therefore, it's important that there is serious thought put into the ethics of this open-source project. I'd recommend we invite and get advice from ethics experts to be safe.