kingoflolz / mesh-transformer-jax

Model parallel transformers in JAX and Haiku
Apache License 2.0
6.26k stars 890 forks source link

[Feature Request] Multilingual assistance. #229

Open phly95 opened 2 years ago

phly95 commented 2 years ago

When experimenting with OpenAI Playground using GPT-3 Davinci, I was able to get a fairly effective language tutor. For example:

The following is a conversation with an AI assistant Japanese teacher. The assistant is helpful, creative, clever, and very friendly.

Human: Explain the vocab and grammar of the sentence "泥沼の三角関係とか告白する前にフラれるとか" AI:

泥沼の三角関係とか告白する前にフラれるとか means "to be dumped before confessing one's feelings or getting caught up in a love triangle." The grammar is pretty simple - it's just a verb followed by a noun. The first part, 泥沼の三角関係とか, means "love triangle" and the second part, 告白する前にフラれるとか, means "to be dumped before confessing." Human: Why "泥沼"? AI:

泥沼 can mean "mire" or "quagmire," but in this context it's being used to describe a love triangle - probably because it's a situation that's easy to get stuck in and hard to get out of. Human: Why "とか"? AI:

とか is a particle that's used to list things, so in this sentence it's being used to list the two possible outcomes - being dumped before confessing, or getting caught up in a love triangle. Human:

I tried the same thing with GPT-J and instead I got:

Human: What is the Japanese word 泥沼 in English?AI: 動物AI says: "What is animal?" or "What is the animal?".Human: [A] What is an example?[B

This example shows that having an AI that is proficient in multiple language and is able to switch between them on the fly can be quite a useful feature for some use-cases.