Build and explore efficient retrieval-augmented generative models and applications
![PyPI - Version](https://img.shields.io/pypi/v/fastrag) ![PyPI - Downloads](https://img.shields.io/pypi/dm/fastrag) :round_pushpin: Installation • :rocket: Components • :books: Examples • :red_car: Getting Started • :pill: Demos • :pencil2: Scripts • :bar_chart: BenchmarksfastRAG is a research framework for efficient and optimized retrieval augmented generative pipelines, incorporating state-of-the-art LLMs and Information Retrieval. fastRAG is designed to empower researchers and developers with a comprehensive tool-set for advancing retrieval augmented generation.
Comments, suggestions, issues and pull-requests are welcomed! :heart:
[!IMPORTANT] Now compatible with Haystack v2+. Please report any possible issues you find.
DocumentStore
support.For a brief overview of the various unique components in fastRAG refer to the Components Overview page.
LLM Backends | |
Intel Gaudi Accelerators | Running LLMs on Gaudi 2 |
ONNX Runtime | Running LLMs with optimized ONNX-runtime |
OpenVINO | Running quantized LLMs using OpenVINO |
Llama-CPP | Running RAG Pipelines with LLMs on a Llama CPP backend |
Optimized Components | |
Embedders | Optimized int8 bi-encoders |
Rankers | Optimized/sparse cross-encoders |
RAG-efficient Components | |
ColBERT | Token-based late interaction |
Fusion-in-Decoder (FiD) | Generative multi-document encoder-decoder |
REPLUG | Improved multi-document decoder |
PLAID | Incredibly efficient indexing engine |
Preliminary requirements:
To set up the software, install from pip
or clone the project for the bleeding-edge updates. Run the following, preferably in a newly created virtual environment:
via pip
pypi:
pip install fastrag
or from a local clone:
pip install .
There are several dependencies to consider, depending on your specific usage (also works with pip install fastrag[*]
package):
# Additional engines/components
pip install .[intel] # Intel optimized backend [Optimum-intel, IPEX]
pip install .[elastic] # Support for ElasticSearch store
pip install .[qdrant] # Support for Qdrant store
pip install .[colbert] # Support for ColBERT+PLAID; requires FAISS
pip install .[faiss-cpu] # CPU-based Faiss library
pip install .[faiss-gpu] # GPU-based Faiss library
# Development tools
pip install .[dev]
The code is licensed under the Apache 2.0 License.
This is not an official Intel product.