Closed nikhilanayak closed 3 years ago
GeDi can be plugged into GPT-Neo since they use the same tokenization, however if you just try to load GPT-Neo in place of GPT-2 I suspect the code will fail to run because GPT-Neo is only supported in newer versions of huggingface and the GeDi code uses an older version. So it will likely require some slight modifications to make them compatible.
As for your follow-up, it's not currently possible to combine attributes in that way with this codebase.
I just checked out the blog post and GeDi seems like exactly what I need for what I'm working on. However, I need generation similar to the levels that are found with GPT3 through an open-source model. Is there support for GPT-Neo?
Edit: Followup question: Is it possible for generation to have multiple topics. For example, you might have, positive, negative, sports, and news, so you could do positive news, negative news, etc.?