Open amirfz opened 1 year ago
commenting to participate
I would like to contribute to developing this tool.
I would like to participate
Meeting Note of Aug 25
Fine tune a hugging face model to create a QA tool.
To this end our work group finds the following learning resources:
https://www.deeplearning.ai/short-courses/finetuning-large-language-models/ https://medium.com/@yuhongsun96/host-a-llama-2-api-on-gpu-for-free-a5311463c183
https://towardsdatascience.com/deploying-large-language-models-with-huggingface-tgi-981747c669e3
https://huggingface.co/learn/nlp-course/chapter7/7?fw=pt
https://github.com/langchain-ai/langchain
evaluate existing hugging face model and choose which base model to fine tune.
Fine tune a chosen hugging face model
list evaluation metrics(helpfulness, truthfulness, perplexity) and how to measure them
Create a LLM Q&A application using LangChain
Where is the data?: The course provides some training data, we can get more data from Amir.
great opening, team
1 & 2 do not require fine tuning and can get us 80% of the way toward our goal for the first release.
Hugging Face has a Repo card API to get description which is a great place to start.
Hey! :)
Sept 1 Meeting
Team is learning the repo API
Tools:
Data:
Web scraping take time. We can feed to vector database like pinecone incrementally.
Alternative is to use the first search result of google as context to sherpa
Question: give me an example of finance data set Question: Does hugging face has a chat optimized version of Llama 2? Question: What is the cheapest model for ___ application?
@yungchidanielcho et al
Alternative is to use the first search result of google as context to sherpa
this already exists through the serpi tool. no need to recreate it
also did you look at the APIs instead of scraping?
Hi @amirfz @yungchidanielcho @Aminolroaya I’m the maintainer of LiteLLM (abstraction to call 100+ LLMs)- we allow you to create a proxy server to call 100+ LLMs, and I think it can solve your problem (I'd love your feedback if it does not)
Try it here: https://docs.litellm.ai/docs/proxy_server https://github.com/BerriAI/litellm
import openai
openai.api_base = "http://0.0.0.0:8000/" # proxy url
print(openai.ChatCompletion.create(model="test", messages=[{"role":"user", "content":"Hey!"}]))
Ollama models
$ litellm --model ollama/llama2 --api_base http://localhost:11434
Hugging Face Models
$ export HUGGINGFACE_API_KEY=my-api-key #[OPTIONAL]
$ litellm --model claude-instant-1
Anthropic
$ export ANTHROPIC_API_KEY=my-api-key
$ litellm --model claude-instant-1
Palm
$ export PALM_API_KEY=my-palm-key
$ litellm --model palm/chat-bison
I'm interested in creating a question-answering tool using hugging face model.