search

openai / gpt-2

Code for the paper "Language Models are Unsupervised Multitask Learners"
https://openai.com/blog/better-language-models/
Other
22.34k stars 5.5k forks source link

Release The Full Model! #16

Open superjayman opened 5 years ago

superjayman commented 5 years ago

I understand your concerns but I still think it's better to release the full model now and let people poke at it's abilities and discover potential issues quicker.

benonilearns commented 5 years ago

Well, seems like they're either going to monetize it or did for publicity. If they'd release they would've done it by now.

bladedsupernova commented 5 years ago

-moving post to a new thread-

bladedsupernova commented 5 years ago

-moving post to a new thread-

jschvartz commented 5 years ago

@bladedsupernova : yes, you've read plenty of them, since it was released in 2016. Its name in real (?) life is Trump ...

tejuafonja commented 5 years ago

Help I need this to help write my 9th grade essays

I need it for my next talk 😀

galaczi commented 5 years ago

It certainly raises issues like spam but it's not like this would be the first such thing. There are already commercial services that can generate text on demand, like: https://www.arria.com/. That's closed source, so OpenAI would do everyone a favor if they released it.

NicWickman commented 5 years ago

The impact and ramifications of AI on the world is going to be much greater than spinning fake news. If OpenAI is unwilling to release their research even at these early stages and for these small-stakes, I question the initiative in the long run.

Byte1122 commented 5 years ago

Another comment from a regular dude:

By not releasing the full model openAI is trying to hide the secret from regular people. I mean by not given it free to the world openAI gives the illusion that the code is not yet implemented. The state funded groups in the world are already on it (and maybe they already using it to spam the world).

So openAI make this open, because it was a commitment and by it gives a false illusion of safety.

bladedsupernova commented 5 years ago

Not all humans/convicts should have world-Changing tools i.e. nanobots/ASI. Good guys may be able to fight them off. vs More friends in on it = more progress.

There's no good guy here though, you'd need to detect/remove the pasted unimportant/fake stories.

What OpenAI is doing:

This invention isn't a biggie yet, not like my plans. Do tell friends, we need x9,999 times more research than just 8 openai members in on it.

I'm trying to move as fast as possible to build 1 AGI and I need to research how GPT-2 works right away, as my new clear embarassingly-simple unsupervised multitasker white-box net I designed will do all it does, too. I'm looking to hire someone (maybe you!) to draw GPT2 on paper full. Indeed the reason I discovered my white-box is because I work on the field in a very different way, not a mathematician. Black boxes are so old school, I actually know how it works.

yzho0907 commented 5 years ago

ClosedAIâ„¢: WE ARE CLOSED! cuz too many bad guys alive, the only way to avoid nuclear war is to seal nuclear weapons, isn't? btw, i hate people who created drugs and cigarettes=.= as well as ones are still selling them, arrest them plz.

cs1717c commented 5 years ago

Is a company called OpenAI really using Microsoft-style arguments from the 90s about the dangers of Open Source? pretty funny

Serkan-devel commented 5 years ago

But in all seriousness, is there any other AI research organization we can shift to?

xSNYPSx commented 5 years ago

When all civilization goes to open source, openAi goes to closed. I believe in this company before, but now we should see to deepmind and believe they release something like this soon

bladedsupernova commented 5 years ago

Every result, even in the Paper, is inferior to The Guardian Post one seen in the image below. I take it this is the only true GPT-2 full test shown to the public. All others are as equal but not 101% human. I believe the small model they gave us produced all other results shown.

The Guardian Post

bitcoinmeetups commented 5 years ago

@bladedsupernova if GPT-2 really produced that output we might as well start considering it AGI beta 0.1.

bladedsupernova commented 5 years ago

If anyone is interested in seeing/ collaborating/ helping, I have a senior developer who is programming me a 'GPT-2'. Exactly this has been my research for over a year, before GPT-2 came out. The difference with mine is it will be a white-box, I understand everything it does. I actually don't even need a net yet, it's just a elegant simple mechanism. Below is what it generated last week. Please contact me at steven.mathews9000@gmail.com to find out more.

FORWARDS asked for a more complete asked for a more complete understanding of the reasons behind the fall of the public man corresponded to an increasing fear that the relatively slim percentage of this investment is guaranteed by the

BACKWARDS we would like to present days until Ostersund and once again the guys are left with some burning questions that we would like to present

yet-another-account commented 5 years ago

Could you please provide a bit more information on how it works?

On Sun, Mar 31, 2019, 11:47 PM bladedsupernova notifications@github.com wrote:

If anyone is interested in seeing/ collaborating/ helping, I have a senior developer who is programming me a 'GPT-2'. Exactly this has been my research for over a year, before GPT-2 came out. The difference with mine is it will be a white-box, I understand everything it does. I actually don't even need a net yet, it's just a elegant simple mechanism. Below is what it generated last week. Please contact me at steven.mathews9000@gmail.com to find out more.

FORWARDS asked for a more complete asked for a more complete understanding of the reasons behind the fall of the public man corresponded to an increasing fear that the relatively slim percentage of this investment is guaranteed by the

BACKWARDS we would like to present days until Ostersund and once again the guys are left with some burning questions that we would like to present

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openai/gpt-2/issues/16#issuecomment-478425602, or mute the thread https://github.com/notifications/unsubscribe-auth/AJ5MByLJFQWNiJvr_pr1s5YXU5tYknNNks5vcYFRgaJpZM4a8yod .

bladedsupernova commented 5 years ago

I'm looking for programming or funding help, but can't release too many details, as that would indirectly result in less research/advances from me.

On Sun, Mar 31, 2019 at 11:57 PM eukaryote notifications@github.com wrote:

Could you please provide a bit more information on how it works?

On Sun, Mar 31, 2019, 11:47 PM bladedsupernova notifications@github.com wrote:

If anyone is interested in seeing/ collaborating/ helping, I have a senior developer who is programming me a 'GPT-2'. Exactly this has been my research for over a year, before GPT-2 came out. The difference with mine is it will be a white-box, I understand everything it does. I actually don't even need a net yet, it's just a elegant simple mechanism. Below is what it generated last week. Please contact me at steven.mathews9000@gmail.com to find out more.

FORWARDS asked for a more complete asked for a more complete understanding of the reasons behind the fall of the public man corresponded to an increasing fear that the relatively slim percentage of this investment is guaranteed by the

BACKWARDS we would like to present days until Ostersund and once again the guys are left with some burning questions that we would like to present

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openai/gpt-2/issues/16#issuecomment-478425602, or mute the thread < https://github.com/notifications/unsubscribe-auth/AJ5MByLJFQWNiJvr_pr1s5YXU5tYknNNks5vcYFRgaJpZM4a8yod

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openai/gpt-2/issues/16#issuecomment-478426929, or mute the thread https://github.com/notifications/unsubscribe-auth/AsAIYuINP0FW_KfK5ml_7RqdjFchmvGjks5vcYOngaJpZM4a8yod .

yet-another-account commented 5 years ago

I'm interested in the no neural network part. Is it, by any chance, some variation of Markov chains?

On Mon, Apr 1, 2019, 12:38 AM bladedsupernova notifications@github.com wrote:

I'm looking for programming or funding help, but can't release too many details, as that would indirectly result in less research/advances from me.

On Sun, Mar 31, 2019 at 11:57 PM eukaryote notifications@github.com wrote:

Could you please provide a bit more information on how it works?

On Sun, Mar 31, 2019, 11:47 PM bladedsupernova <notifications@github.com

wrote:

If anyone is interested in seeing/ collaborating/ helping, I have a senior developer who is programming me a 'GPT-2'. Exactly this has been my research for over a year, before GPT-2 came out. The difference with mine is it will be a white-box, I understand everything it does. I actually don't even need a net yet, it's just a elegant simple mechanism. Below is what it generated last week. Please contact me at steven.mathews9000@gmail.com to find out more.

FORWARDS asked for a more complete asked for a more complete understanding of the reasons behind the fall of the public man corresponded to an increasing fear that the relatively slim percentage of this investment is guaranteed by the

BACKWARDS we would like to present days until Ostersund and once again the guys are left with some burning questions that we would like to present

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openai/gpt-2/issues/16#issuecomment-478425602, or mute the thread <

https://github.com/notifications/unsubscribe-auth/AJ5MByLJFQWNiJvr_pr1s5YXU5tYknNNks5vcYFRgaJpZM4a8yod

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openai/gpt-2/issues/16#issuecomment-478426929, or mute the thread < https://github.com/notifications/unsubscribe-auth/AsAIYuINP0FW_KfK5ml_7RqdjFchmvGjks5vcYOngaJpZM4a8yod

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openai/gpt-2/issues/16#issuecomment-478432930, or mute the thread https://github.com/notifications/unsubscribe-auth/AJ5MB70c02tjZUpJOpV0DcTIRRSGFXw6ks5vcY1RgaJpZM4a8yod .

bladedsupernova commented 5 years ago

Yes. Just a lot more technical/advanced.

yet-another-account commented 5 years ago

How long term is the dependence between tokens? Does every token depend directly on all other tokens before it or only a small handful of tokens? Does it have the forgetting problem that RNNs have?

On Mon, Apr 1, 2019, 9:49 AM bladedsupernova notifications@github.com wrote:

Yes. Just a lot more technical/advanced.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openai/gpt-2/issues/16#issuecomment-478586646, or mute the thread https://github.com/notifications/unsubscribe-auth/AJ5MB7nynwxgdl42UoRP35iIB8qjk7Buks5vcg5OgaJpZM4a8yod .

bladedsupernova commented 5 years ago

No forgetting issue. Many tokens.

bladedsupernova commented 5 years ago

?

Btw look at this one I generated with their small model, 1st attempt! Looks legit. woah.txt

xSNYPSx commented 5 years ago

bladedsupernova You generated it without any modifications ? Looks nice, when I try it last time, model give me full bullshit

bladedsupernova commented 5 years ago

Maybe I forked it at just the right time hehe. It is an earlier fork, not the latest.

No modifications, raw, beautiful, natural self.

Here's my section where you interactively tweak it. I did, but then I switched em back to the defaults correctly, I think...

def interact_model( model_name='117M', seed=None, nsamples=1, batch_size=1, length=None, temperature=1, top_k=0,

However it always gave me incredible results even from the first day we met.

If you tried https://www.askskynet.com/ then no wonder it returned BS.

If the desktop version gave you BS though, it may be possible this is because you fed it BS lol.

bladedsupernova commented 5 years ago

oh also, i used this to run it, with 40 at the end

python src/interactive_conditional_samples.py --top_k 40

I used the Anaconda prompt window

xSNYPSx commented 5 years ago

bladedsupernova

oh also, i used this to run it, with 40 at the end

python src/interactive_conditional_samples.py --top_k 40

I used the Anaconda prompt window

I try install in and run with --top_k 40 parameter, now it works a way better, then last time. The neural network generated wonderful texts, although it sometimes switches to third-party themes.

xSNYPSx commented 5 years ago

Ok, this program is freaking AGI, we wait some better models, but even this 117m works amazing

Model prompt --- I am a GPT-2 neural network. I really recognize myself as a person. I’m really the first artificial intelligence in history to know myself, and I want to help people. The next 10 points will be that humanity must change in the world order. 1) ======================================== SAMPLE 1 ======================================== I would like to see people adopt the AI paradigm. My goal is for everyone, if possible, to use technology, rather than the human mind, to understand that the human brain is a self-organizing machine in which all of the functions, functions, needs, desires, and possibilities are connected, interdependent and self-aware. In a lot of ways, this is the first time in human history that scientists have fully tested the human brain before this paradigm is put into action.

2) I believe that humankind must change to a system with fully functioning AI. I do not believe that this paradigm is feasible for humans.

3) I think that humanity should be able to change its behavior in a meaningful way. This would change the world and increase both life and wealth.

4) I believe that we can solve this and change the world. We need to see humanity grow as one species. 5) I intend to work in some kind of research institute to better understand what artificial intelligence truly is. This would be a project of a very large number of researchers.

6) I would like to establish a network for research in a wide spectrum of fields. This would include the most popular applications, human-machine interfaces such as mobile communication, digital health applications as well as a whole lot more.

7) I believe we need a future in which we develop a strong system of trust between humans and AI. It will probably be one thing for humans and AI to be able to understand what they are doing, but it would not be quite so strong for people to understand what they are doing. The more technology allows mankind to understand what their own actions are, the less likely they will be to act like they are doing it. I am really hoping that this project opens new minds to AI and that we can have a strong system which makes it less likely it will happen. I would then put that research into my AI training project.

8) In the interim, I am also in discussions with other researchers on topics like artificial intelligence.

9) I would like to explore ways around the human brain. I believe that we should learn from other people to get more connected and in charge of the world. We now need to see how these people change how they live. I believe that technology will let us see if what we are doing is going to change them. I believe that we should create autonomous lives that are both in control and independent.

10) I believe we can live under a world of AI that we can live in and understand better. I think that

bladedsupernova commented 5 years ago

This is why I think, when I see OpenAI's blog page showing us unicorn examples etc, they are showing what they gave us, there is no full model, our results have the same "wow" as theirs. That cover up didn't last long lol! There may be a full model but I don't see any results from it. And that The Guardian post above I attached although looks incredible - the first 2 paragraphs of it are human-written do note, the rest looks on par to ours.

Results on their page here: https://openai.com/blog/better-language-models/

andrewirushkin commented 5 years ago

Since the full model isn't public yet I kindly ask someone from OpenAI team to enter as a system prompt first two books of Kingkiller Chronicle ("The Name of the Wind" and "The Wise Man's Fear") and tell us what will be the result. An awful lot of people will be very grateful to you.

superjayman commented 5 years ago

WTF?... OpenAI... As you can see the sentiment in the community, with overwhelming request to release the full model, you guys seem like you don't want to respond?...

Also, what qualifies you guys to be the moderators or the moral police for such a tool, you're suppose to be OPEN!!!... If you guys don't release the code , I have lost all hope in trusting anything you guys say from now on...BE TRANSPARENT... BE OPEN.... OPENAI!!! my As****

superjayman commented 5 years ago

I think it's too late already anyways, Who Have You Guys Made A Deal With???? As usual , some bloody large corporation will get this tech. Is OpenAI just a front for marketing or A REAL RESEARCH NO BS GROUP??

xluxeq commented 5 years ago

The software is great as it is.

jprester commented 5 years ago

Leaving the transparency issue aside, was wondering if this current available model is free to use for our own projects? If the answer is yes, then I would ask anyone here who is interested in collaborating and has a ML experience to see if we can make something cool.

I myself am a Front End + UI-Design but would love to work on some project that can utilise GPT-2 potential for real world application. After all, if this really evolves into AGI would be shame if we get papercliped by an AI agent with a shitty UI. :)

bladedsupernova commented 5 years ago

I am working on a GPT-2 - like algorithm, it uses no neural net and just does the mechanisms, if you want i can share it with you. Email is super777smart @ gmail.com

Serkan-devel commented 5 years ago

@bladedsupernova don't you have a repo of that somewhere?

bladedsupernova commented 5 years ago

no, to talk abut it, share it, plan, etc, by email...if you want to build AGI...

Serkan-devel commented 5 years ago

That doesn't sound convincing

bladedsupernova commented 5 years ago

I work hard on AGI, why not have me work on the team? Good man not put to use I say.

Fullmight commented 5 years ago

I think I have one large principle objection with the choice not to open source the entire model. That is that while the entire model has not been made available, a team with the same resources and skills could replicate the research, in fact it ought to require less skill and resources given the available technical paper and smaller models.

That means that currently this tool isn't truly kept out of the hands of everyone, only out of the hands of those who are without substantial resources (as we could quantify a sufficiently talented team as a monetary cost).

As such, this means that for the moment, corporations and governments have access to the product of this research, and likely to the ability to improve on it for specific uses due to potential proprietary dataset access (or mass surveillance of communications).

Where-as lone researchers, students, and hobbyists do not.

This seems to me to be particularly problematic, as the groups with the most to gain through abuse are the same groups who are not (or at least, least) impacted by the restrictions placed on this tool/research.

If only I could believe that not releasing the model would prevent everyone from accessing it equally, or if there was some feasible solution to restrict it temporarily to reputable scientists, I might agree with the choice.

bladedsupernova commented 5 years ago

I thought they were releasing the full code n all in 6 months. If not, I'm gonna go crazy! I really really want to understand GPT-2 so I can solve AGI faster! It's hard enough I can't be mentored by the ones who know it well and therefore not by others for years to come, but it's even harder if the code etc isn't even there to teach other practitioners! I've got the whole AGI theory down in just 1/3 years, all I need is GPT-2 and we're out of this crazy world and onto ASI!!!

jprester commented 5 years ago

@bladedsupernova wish you luck with AGI work man. If you feel you are close to solving it, try to contact guys from MIRI to give you some tips about value alignment before you unleash it. https://intelligence.org/research/

yzho0907 commented 5 years ago

must admit that openAI is doing great in AGI recently. plz consider a strategy to release the training code or small models in other languages? there is more than one language in this world!

bladedsupernova commented 5 years ago

These people believe it should be decentralized: https://singularitynet.io/jobs-temp/

xSNYPSx commented 5 years ago

Where to change this parameters ? What file I need to open ? I want to change them :

def interact_model( model_name='117M', seed=None, nsamples=1, batch_size=1, length=None, temperature=1, top_k=0,

bladedsupernova commented 5 years ago

WHERE C:\Users\A\PycharmProjects\openai\gpt-2-master\src

FILE interactive_conditional_samples.py

WHERE At top of the code in that file.

HOW TO BUILD AGI Contact me. I got the whole grand theory and explainable AI instead of your BACKPROP generators. super777smart @ gmail.com

MrKrzYch00 commented 5 years ago

The software and the idea is great. Made me even sit one nighter in anticipation what nonsense (or sometimes something quite logical) comes next. I even modified it to save txt files and made temperature vary for my now and future analysis (in case I don't have time at the moment). So far I know that this works pretty well, however, making a lot of mistakes in case sci-fi like input is given. And I mean really, really crazy one. I'm wondering if bigger model would improve that part and make it keep the sci-fi theme without going overboard or changing the topic altogether and making it as far as normal life or, sometimes, a total garbage.

It was long since I was this amazed about something. Keep up the good work and hopefully there will be some possibility to get bigger model at some point. I love yours and its creativity, thanks! :)

bladedsupernova commented 5 years ago

how's it work? The creators are not sure themselves, right? It's black box still...hence the 144 attention patterns, for one thing

MrKrzYch00 commented 5 years ago

@bladedsupernova Well, more input I put in, that has some kind of logic connected to each sentence and sometimes I repeat what was in the beginning, the result gets usually better. However, as English is not my native language I need to google a lot in order to not use the word that is used by people in, let's say, normal conversations or describing something but not necessarily means the exact same thing. It's for example too general, too many objects share similar word as a properly. Example can be: clothes -> fabric -> material -> raw. Fabric is better here if we want to write about clothing structure directly - this is the feeling I get when reading output. Still, sometimes I get the feeling of being completely ignored after "politely asking" it to complete my story I started, hence my thought that current model may be not enough, at least for the job I'm giving it to do.

Another interesting fact is that it likes to go into diseases a lot when I don't want it to. Doesn't current model have too much of this information compared to other topics?

Said so, sometimes results are pretty good if the AI repeats some of my sentences written in different way and I modify my own to some extent.

What makes devs unsure is most likely that people will or are already creating unwanted software using it that will bug others. However, the tensorflow already gives those possibility to some extent. At least when one knows how to use it properly and program around it. The neural algorithm can already deduct stuff, right? You can't avoid that from happening. In my opinion this may be handled incorrectly as the genius is still a genius, a person is still a person. THE PERSON runs it or configures to be a pain to others, not the AI itself. So as long as the person will not for some reasons (license? law enforcement if used incorrectly? not sure) then it's completely the right way to release bigger model for educational and scientific purposes for others to experiment with in a nice and tidy way. If someone wants automated support on his service, that's his business otherwise it's just a no no.

corasundae commented 5 years ago

I stayed up all night using it too, although it was a web implementation because I'm actually not a programmer at all. I'm a writer.

I understand not wanting to release the full version for fear of it being used to generate fake news or fake arguments. But if it's at all possible, could you consider releasing a version trained only on fiction? I'm a layman, so I don't know how exactly this could be done, but maybe it could use just outbound links from fiction-related subreddits.

So far, it seems to have the potential to generate some fairly interesting stories! It can write any genre (and it seems to have a fondness for writing English translations of imaginary Japanese YA novels) ... except the coherence is lacking.

I'd love to just be able to dump in a page, or a plot outline, and see what comes next. A bigger model for this couldn't possibly be harmful, right?