search

Aggregate-Intellect / sherpa

https://sherpa-ai.readthedocs.io/
Other
165 stars 45 forks source link

create a hugginface QA tool #64

Open amirfz opened 1 year ago

yungchidanielcho commented 1 year ago

I'm interested in creating a question-answering tool using hugging face model.

bharatji30 commented 1 year ago

commenting to participate

Aminolroaya commented 1 year ago

I would like to contribute to developing this tool.

digitalpaddyo commented 1 year ago

I would like to participate

yungchidanielcho commented 1 year ago

Meeting Note of Aug 25

Objective

Fine tune a hugging face model to create a QA tool.

To this end our work group finds the following learning resources:

https://www.deeplearning.ai/short-courses/finetuning-large-language-models/ https://medium.com/@yuhongsun96/host-a-llama-2-api-on-gpu-for-free-a5311463c183

https://medium.com/the-techlife/using-huggingface-openai-and-cohere-models-with-langchain-db57af14ac5b

[https://cobusgreyling.medium.com/langchain-creating-large-language-model-llm-applications-via-huggingface-192423883a74]

https://thealgorithmicminds.com/how-to-use-huggingface-to-use-llama-2-on-your-custom-machine-35713a2964de

https://towardsdatascience.com/deploying-large-language-models-with-huggingface-tgi-981747c669e3

https://huggingface.co/learn/nlp-course/chapter7/7?fw=pt

https://github.com/langchain-ai/langchain

Tasks break down

  1. evaluate existing hugging face model and choose which base model to fine tune.

  2. Fine tune a chosen hugging face model

  3. list evaluation metrics(helpfulness, truthfulness, perplexity) and how to measure them

  4. Create a LLM Q&A application using LangChain

By next meeting we will:

Where is the data?: The course provides some training data, we can get more data from Amir.

amirfz commented 1 year ago

great opening, team

yungchidanielcho commented 1 year ago

Aug 29 dev's meeting

Amir's insight

  1. scrap Hugging Face model description (not the model itself), feed to vector database
  2. chunk the description text to improve vector database search result
  3. fine tune only if necessary

1 & 2 do not require fine tuning and can get us 80% of the way toward our goal for the first release.

hugging face API

Hugging Face has a Repo card API to get description which is a great place to start.

house keeping

mei-chen commented 1 year ago

Hey! :)

yungchidanielcho commented 1 year ago

Sept 1 Meeting

Repo API

Team is learning the repo API

Web scraping

Tools:

Data:

Web scraping take time. We can feed to vector database like pinecone incrementally.

Do we want to have a database with complete knowledge of HF?

Alternative is to use the first search result of google as context to sherpa

Use case exploration: What are the questions we want to ask sherpa?

Question: give me an example of finance data set Question: Does hugging face has a chat optimized version of Llama 2? Question: What is the cheapest model for ___ application?

amirfz commented 1 year ago

@yungchidanielcho et al

Alternative is to use the first search result of google as context to sherpa

this already exists through the serpi tool. no need to recreate it

also did you look at the APIs instead of scraping?

ishaan-jaff commented 11 months ago

Hi @amirfz @yungchidanielcho @Aminolroaya I’m the maintainer of LiteLLM (abstraction to call 100+ LLMs)- we allow you to create a proxy server to call 100+ LLMs, and I think it can solve your problem (I'd love your feedback if it does not)

Try it here: https://docs.litellm.ai/docs/proxy_server https://github.com/BerriAI/litellm

Using LiteLLM Proxy Server

import openai
openai.api_base = "http://0.0.0.0:8000/" # proxy url
print(openai.ChatCompletion.create(model="test", messages=[{"role":"user", "content":"Hey!"}]))

Creating a proxy server

Ollama models

$ litellm --model ollama/llama2 --api_base http://localhost:11434

Hugging Face Models 

$ export HUGGINGFACE_API_KEY=my-api-key #[OPTIONAL]
$ litellm --model claude-instant-1

Anthropic 

$ export ANTHROPIC_API_KEY=my-api-key
$ litellm --model claude-instant-1

Palm

$ export PALM_API_KEY=my-palm-key
$ litellm --model palm/chat-bison