search

patterns-ai-core / langchainrb

Build LLM-powered applications in Ruby
https://rubydoc.info/gems/langchainrb
MIT License
1.42k stars 195 forks source link

Replace `hugging-face` gem with `transformers-ruby` gem #792

Open andreibondarev opened 1 month ago

andreibondarev commented 1 month ago

Description

The hugging-face gem only allows us to call the huggingface.co Inference API and we're unable to pull down models locally and run them on local machines.

We should be able to use @ankane's new transformers-ruby library to pull down and run models locally.

Tasks:

  1. Replace hugging-face gem with transformers-ruby
  2. Pick out suitable models to be used as default models for def embed, def complete and def chat.
  3. Build out def chat() method. You should be able to use chat templates and pass an array of messages: [{...}, {...}, ...]. Additional information on Chat Templates.
  4. Modify def embed(text:, model:) method to work with the new underlying transformers-ruby library.
  5. Modify def complete(prompt:, model:) method to work with the new underlying transformers-ruby library.
ankane commented 1 month ago

Hey @andreibondarev, Informers might be a better fit for this (or useful as another option), as it's much easier to install (no need for LibTorch and a compiler). Also, it supports the text-generation pipeline (although I've only tried it with GPT-2, the default model).