Migrated to Groq with Sentence Transformers

[hriteshMaikap]

I have added the feature as requested, but with some major changes. The issue of delayed responses has beeen resolved, now the only delay is loading the cache files, which takes about 20 seconds. After which responses are very fast (refer to the attached video). Let me help summarize the files. I have not changed you main_local.py instead created a updated_main_local.py with some supporting files namely config.py and .env. But why did I do this? Here is the explaination:

• I have used sentence-transformers instead of OllamaEmbeddings which are quicker and more lightweight to store with FAISS Indexing
• The new implementation saves the embeddings and FAISS index locally in a specified cache directory. This allows for faster access during subsequent queries, significantly reducing load times and improving overall user experience.
• I noticed that the requirement is Ollama for using LLama3.1 for the generation of answers. I figured it is too heavy to operate on machines, hence I have switched to FAST AI Inference Engine GROQ. This not only allows me to access Llama but also other models like Mistral, which further enhances the speed of response. The LLM model being used is still the same, but I am leveraging API calls instead of overloading my machine (handling ollama was a hassle)
• pip install sentence-transformers langchain-groq python-dotenv faiss-cpu will make sure there are no dependancy issues. You just need to login to GROQ to create a API KEY and store it in .env file. The code will then work like a charm.

https://github.com/user-attachments/assets/b661e4d1-de4b-4503-b715-ee53f16d2a46

[hriteshMaikap]

Some known warnings in the console that may raise a few questions:

• RuntimeWarning: Found Intel OpenMP ('libiomp') and LLVM OpenMP ('libomp') loaded at the same time. Both libraries are known to be incompatible and this can cause random crashes or deadlocks on Linux when loaded in the same Python program. ->While this may not cause any errors in development but this will have to be looked during the production. We need to check for compatibilties of libraries.
• oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. ->This is typical warning from what I know
• WARNING:tensorflow:From ... The name tf.losses.sparse_softmax_cross_entropy is deprecated. -> A function that is being used is deprecated, will have to chnage to latest version according to the library needs.

[bakayu]

I have reviewed the changes, merging your PR. I have messaged you on discord as well, please check them out, also I will be doing some changes here (going forward with your approach but keeping the local-llm capabilities for testing purpose).

Thank you for the contribution!! :)