search

minimaxir / aitextgen

A robust Python tool for text-based AI training and generation using GPT-2.
https://docs.aitextgen.io
MIT License
1.84k stars 220 forks source link

Add a "include prompt" paramater to the generate function analagous to gpt-2-simple's #150

Closed johnnymcmike closed 3 years ago

johnnymcmike commented 3 years ago

Hi, I'm working on a discord bot using this project, and I think it would be useful for there to be an option to exclude the text in the prompt parameter of generate from the returned output. I did something similar with gpt-2-simple in another bot that used prefix and include_prefix, and it would be great to have something like it here.

minimaxir commented 3 years ago

That might be better to do outside aitextgen by using the length of the prompt, e.g.

text = text[len(prompt):]

IIRC the include_prefix was more of a hack rather than good design.

johnnymcmike commented 3 years ago

Gotcha. I had just figured that it was somehow implemented elegantly behind the scenes (particularly in a way that lets you ignore the length of the prefix when setting a length of text to generate, saving unnecessary generation time). If it was pretty much the same hack that I now use in its stead, though, then I'll just keep using that lol. Thank you!

minimaxir commented 3 years ago

Essentially that part of the generation is the same (the prefix/prompt is first encoded, then generated from there)

I may add it back since there are valid normal use cases for it.