search

FMInference / FlexLLMGen

Running large language models on a single GPU for throughput-oriented scenarios.
Apache License 2.0
9.18k stars 548 forks source link

Where is the chatbot? I miss it! #87

Closed fuhengwu2021 closed 1 year ago

fuhengwu2021 commented 1 year ago

Why delete it?

brandonj60 commented 1 year ago

Agreed. They also deleted all the readme instructions for some reason.

Ph0rk0z commented 1 year ago

Use https://github.com/oobabooga/text-generation-webui

Nidelon commented 1 year ago

It's not too good to remove all instructions from the README and example files, while not adding docs. This is extremely confusing for beginners and generally complicates the understanding of the library.

SoftologyPro commented 1 year ago

Add another request to bring the chatbot back, and if possible get it to be able to handle multiple line responses.

fuhengwu2021 commented 1 year ago

Also is it able to use 2 GPUs with FlexGen and how?

cidrugHug8 commented 1 year ago

Perhaps they removed it for fear of abuse.

Nidelon commented 1 year ago

Perhaps they removed it for fear of abuse.

It is unlikely, since there was just an input by the type of Person: Bot:, nothing special. And to think about the abuse of this library is slightly stupid, just as it is possible for the same reasons, for example, to prohibit the use of Google Colab.