search

TheProtaganist / gpt-j

A GPT-J API to use with python3 to generate text, blogs, code, and more
MIT License
205 stars 52 forks source link

How do I include a stop sequence? #2

Open amrezo opened 3 years ago

amrezo commented 3 years ago

Hello, I am creating a chatbot with this API. I was wondering how I can add a stop sequence so it doesn't keep on generating text after the response.

Thank you very much!

TheProtaganist commented 3 years ago

In a way the the "user" parameter is the stop token. I had to create it manually because the server I'm using for it provided by vicgalle doesn't have a stop token. What you can do is define the user and bot parameters. The user is the name of who the bot is speaking to, the bot is the imaginary person or thing you are planning to speak to. I highly recommend you don't use periods in the examples. It causes the model to continue when it wasn't supposed to... Just make sure in the examples the user prompts are on the left and the bot examples are on the right

TheProtaganist commented 3 years ago

The server sometimes goes down. If the API doesn't seem to respond it's probably because the website got updated or something. Is ok they will have it fixed soon.

TheProtaganist commented 3 years ago

The website is up again. If you don't mind can you tell me what kinda chatbot your gonna build 🙂. Is it for business, personal, or something else?

TheProtaganist commented 3 years ago

Good news stop token was just added to the server! I will implement it and let you know.

Simplicitylinux commented 3 years ago

I'd be interested in being able to define a stop sequence too. If you've implemented it, could you let me know?

bensonchow123 commented 3 years ago

Me too, i wish stop sequences are implemented! It can finally become an alternative to gpt-3, if stop sequence are implemented!

vicgalle commented 3 years ago

@Simplicitylinux @bensonchow123 If it is of interest, I have already implemented custom stop sequences at https://github.com/vicgalle/gpt-j-api