Testing LangChain custom tools powering Crypto search engine

[kjain25]

Testing out custom tools to evaluate how accurately they answer client's questions about Crypto, and how often does the engine select the anticipate tool for each question

[kjain25]

Even if all outputs to questions seem to be reliable, we need to ensure the accuracy of the answers. We need to see how often the algorithm is resorting to Google Search vs Coin Gecko API calls, because Coin Gecko has more up-to-date data. We also need to test what Coin Gecko API calls are being used for which question, regardless of the answer, because the agent may be resorting to an inefficient approach for answering the question. Even though multiple API endpoints may be able to answer the same questions, some may be shorter in output and more direct than others. So, it is important that the most efficient and direct custom tool is selected for each question.

[kjain25]

Current blocks: LLMs and LangChain have token limits and even though we are ensuring that the function call outputs don't exceed token limit. Sometimes prompt + completion together can exceed token limit