Closed Berowne closed 3 years ago
Got it to work with rebuild... and pip install faiss and faiss-gpu git clone https://...github.../rag export TOKENIZERS_PARALLELISM=false pip install torch torchvision ray[default] datasets faiss faiss-gpu matplotlib seaborn pandas transformers awscli s3fs scikit-plot python use_own_knowledge_dataset.py --csv_path ./text.csv --output_dir ./out/text
@Divyansh-Gemini This looks more like a pip issue, as faiss is available on pypi. I'd suggest upgrading pip and trying again:
pip install --upgrade pip
pip install faiss
Thanks @amyeroberts for your response.
The issue has been resolved now.
I was installing faiss
, but faiss-cpu
was needed to be installed.
transformers
version: 4.5.1Information
Model I am using (RAG Retriever ...):
The problem arises when using: [ ] the official example scripts: (worked!) [ ] my own modified scripts: (give details below)
The tasks I am working on is: [* ] my own task or dataset: (give details below)
To reproduce
Steps to reproduce the behavior:
Also, if you import faiss, then faiss.version does not exist.
Note for our environment we have to pip install faiss-gpu rather than conda since conda repos are blocked at proxy. qds/NLP/aws_nlp/rag/out
A sample script to query the /path/to/my_knowledge_dataset/ would be handy.