reduce number of llm instances

[SubhadityaMukherjee]

There are too many llm instances now. Because each is it's own service. This is not good and should be re-written. For instance, llama3 is used for deciding what kind of query it is, structured query analysis, and also generating results.

[PGijsbers]

But how do you propose to do this? Create a complex query that does multiple things (at the risk of being more unpredictable)? Replace an LLM by other methods?

[SubhadityaMukherjee]

Well turns out the issue was that the same LLM was being re-initalized multiple times which added to the time taken to get results. (which was the reason this was created) Other than that, it doesnt seem like there is anything else we can do, so I am closing this issue.