search

aws-samples / genai-quickstart-pocs

This repository contains sample code demonstrating various use cases leveraging Amazon Bedrock and Generative AI. Each sample is a separate project with its own directory, and includes a basic Streamlit frontend to help users quickly set up a proof of concept.
Apache License 2.0
315 stars 106 forks source link

sentence-transformers not installed #7

Closed carlosbravoa closed 7 months ago

carlosbravoa commented 8 months ago

I just followed up the quickstart, created the venv, and installed the requirements.txt file As soon as you launch it and start prompting you get an error with sentence-transformers not installed.

So I installed it with: pip install sentence-transformers but ended with a plethora of dependencies installed that are irrelevant to the case such as CUDA (I don't even have an NVIDIA GPU), safetensors, scikitlearn and huggingface-hub. If the models run on Bedrock and this is just APIs, there is something wrong somewhere.

Thanks!

pip install sentence-transformers
...
...
...
Successfully installed huggingface-hub-0.22.2 joblib-1.3.2 mpmath-1.3.0 networkx-3.2.1 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.19.3 nvidia-nvjitlink-cu12-12.4.99 nvidia-nvtx-cu12-12.1.105 safetensors-0.4.2 scikit-learn-1.4.1.post1 scipy-1.12.0 sentence-transformers-2.6.1 sympy-1.12 threadpoolctl-3.4.0 tokenizers-0.15.2 torch-2.2.2 transformers-4.39.2 triton-2.2.0
carlosbravoa commented 8 months ago

And after that, it asked me to install chromadb.

ryanadoty commented 8 months ago

Hey @carlosbravoa , do you mind sharing what sample you are trying to run when you encountered this issue?

kaiyanchen commented 7 months ago

i tried the amazon-bedrock-chat-poc, it works smoothly on M1 with my miniconda env. you'd better to point out which example you tried. @carlosbravoa

carlosbravoa commented 7 months ago

Sorry for the oversight! It was: Amazon-Bedrock-GenAI-Dynamic-Prompting-Explained-POC Thanks!

ryanadoty commented 7 months ago

Hey @carlosbravoa can you try pulling down the latest version of the repo, I believe I addressed this issue :)

Just to clarify the reason you need sentence transformers and chromadb is for two similar but different things. Some of the PoCs are using a dynamic prompting technique, leveraging a semantic similarity search based on a database of stored prompts.

When the app is initially started, it uses sentence-transformers and pulls down an embeddings model from hugging face. This model is then used to create embeddings of all your sample prompts, and stores them in a vector database (ChromaDB) in memory.

Each time you make a query, an embedding of your query is created, you do a KNN search against chromaDB, and it returns the three most semantically similar prompts, and injects those into the final prompt along with your question into Bedrock.

If you wanted, you could replace the hugging face embeddings model with an amazon bedrock based embeddings model, but since the one provided is free, we thought it was the best fit.

Hopefully this clarifies things!