Sorry, this is not a bug, but I wanted some feedback. I've been able to get the Pythia-Chat-Base-7B model running on very low spec hardware (4G gpu) and it works really well, if a bit slow (fair enough), its impressive that it works 'out of the box' on low spec hw (I'm in the process of getting better h/w), without having to chase down obscure python libs or hacking to make it work.
I'm wondering if it can handle multiple conversations at once, eg if it were connected to a
chat system like element, does it require a session for each user or how would it roughly work? Or would it be just one session for anyone in the element channel the bot is in?
Sorry, this is not a bug, but I wanted some feedback. I've been able to get the Pythia-Chat-Base-7B model running on very low spec hardware (4G gpu) and it works really well, if a bit slow (fair enough), its impressive that it works 'out of the box' on low spec hw (I'm in the process of getting better h/w), without having to chase down obscure python libs or hacking to make it work.
I'm wondering if it can handle multiple conversations at once, eg if it were connected to a chat system like element, does it require a session for each user or how would it roughly work? Or would it be just one session for anyone in the element channel the bot is in?
thanks Darrin