search

AlpinDale / pygmalion.cpp

C/C++ implementation of PygmalionAI/pygmalion-6b
MIT License
56 stars 5 forks source link

Infinity generation text #3

Open 4EPBEED opened 1 year ago

4EPBEED commented 1 year ago

Why pygg infinity generation text? The text is't even related to what I wrote.

AlpinDale commented 1 year ago

Why pygg infinity generation text? The text is't even related to what I wrote.

I'm aware of the issue. I've been looking into it.

Problem is, C++ isn't my forte so it'll take a while unless a more skilled person comes along and makes a PR.

4EPBEED commented 1 year ago

Thanks for the answer. Still, your work is amazing. Hope everything works out for you!

4EPBEED commented 1 year ago

I think I understand what the problem of infinite generation is. In the original, Pygmalion catches the "/nYou:" token and stops generation and passes the input to the user. It is necessary either to make changes in the dataset, or to enter a stop list of tokens.

HRNPH commented 1 year ago

I Agree with "/nYou:" token catching It'll be nice to have a one-time inference script too since I think using it through a Python wrapper will be easier just like Tesseract I'm looking forward to implementing this into my AIWaifu_Project Good work btw đź‘Ť

4EPBEED commented 1 year ago

In some cases, I still managed to stop the infinite generation, but not often it didn’t work very well. Where |end of text| it stops there. Now I'm thinking about how to intercept the token to stop and pass the input to the user.