search

Pan-ML / panml

PanML is a high level generative AI/ML development and analysis library designed for ease of use and fast experimentation.
MIT License
113 stars 16 forks source link

Models for text classification & summarisation, and running on CPU #69

Open Dee-Ma opened 1 year ago

Dee-Ma commented 1 year ago

Hi,

Many thanks for releasing this library for using LLMs! We just have two quick questions about models in this library.

  1. Could we know if there are any models that are good for text classification and summarisation tasks?
  2. Could we know if those models can support running locally on CPU for inference?

Thanks!

wanoz commented 1 year ago

Hi thank you for the question, sorry for the late response. For text classification and summarisation, I have personally found value in the Google FLAN T5 models, I know the smaller versions of these models can be run on CPU for inference (such as base, small sizes). The performance of this will vary depending on your text data.