search

nsbradford / SemanticSearch

Minimal RAG (Retrieval Augmented Generation) website with Pinecone, FastAPI, NextJS, MongoDB
https://semantic-search-six.vercel.app
9 stars 3 forks source link

Why use helicone? #12

Open krrishdholakia opened 1 year ago

krrishdholakia commented 1 year ago

https://github.com/nsbradford/SemanticSearch/blob/f8189d5bf05260af95584b0a9d878233b772a234/backend/llm.py#L13

Hey @nsbradford,

I saw you're logging responses to promptlayer but also using helicone. Curious - why?

If it's for caching - is there something in our implementation you think is missing - https://docs.litellm.ai/docs/caching/

nsbradford commented 1 year ago

AFAIK LiteLLM does not support a managed caching solution, the docs only mention self-managed Redis.

krrishdholakia commented 1 year ago

Tracking this here - https://github.com/BerriAI/litellm/issues/432

Do you want us to provide a hosted caching solution? @nsbradford

nsbradford commented 1 year ago

TBH, not a top priority - writing my own cache also only takes <1 hour and then I don't have to worry about whether the 3rd-party caching middleware is reliable, which is why in practice I tend to write my own cache on larger projects

nsbradford commented 1 year ago

(would be open to it if implemented, though.)