Taking a long response time

katanaml / sparrow

Data processing with ML, LLM and Vision LLM

https://katanaml.io

GNU General Public License v3.0

3.73k stars 379 forks source link

Closed SharanyaSarkar closed 9 months ago

SharanyaSarkar commented 9 months ago

The code that you've shared is taking a long time (more than 3 mins) to retrieve the results, how to optimize the response time ?

abaranovskis-redsamurai commented 9 months ago

Main response time comes from LLM execution locally with Ollama. To run it quick, you need to run Ollama on GPU machine or Apple M1 processor.