Can we build a RAG Android application using Local LLM or SLM (HF tflite models)

shubham0204 / Android-Document-QA

A custom RAG pipeline for multi-document QA from PDF/DOCX documents, in Android

Apache License 2.0

55 stars 5 forks source link

Can we build a RAG Android application using Local LLM or SLM (HF tflite models) #7

Open puliviswanath opened 3 months ago

puliviswanath commented 3 months ago

I am exploring the development of a Retrieval-Augmented Generation (RAG) application for Android and am considering using local language models from Hugging Face’s TFLite models. I am looking for guidance and insights on how to effectively integrate these models into an Android application.

vimal-yuvabe commented 1 month ago

Hi @shubham0204 Thanks for this wonderful example of RAG. will u help to add a localized llm using llama.cpp. this will be very much supportive and helpful to the community as whole.

shubham0204 commented 1 month ago

@vimal-yuvabe that's a great idea! I have been working with llama.cpp for a while and will soon integrate it with this project.