search

salesforce / GeDi

GeDi: Generative Discriminator Guided Sequence Generation
https://arxiv.org/abs/2009.06367
BSD 3-Clause "New" or "Revised" License
208 stars 46 forks source link

Is GPT-Neo Supported? #11

Closed nikhilanayak closed 3 years ago

nikhilanayak commented 3 years ago

I just checked out the blog post and GeDi seems like exactly what I need for what I'm working on. However, I need generation similar to the levels that are found with GPT3 through an open-source model. Is there support for GPT-Neo?

Edit: Followup question: Is it possible for generation to have multiple topics. For example, you might have, positive, negative, sports, and news, so you could do positive news, negative news, etc.?

benkrause commented 3 years ago

GeDi can be plugged into GPT-Neo since they use the same tokenization, however if you just try to load GPT-Neo in place of GPT-2 I suspect the code will fail to run because GPT-Neo is only supported in newer versions of huggingface and the GeDi code uses an older version. So it will likely require some slight modifications to make them compatible.

As for your follow-up, it's not currently possible to combine attributes in that way with this codebase.