search

Helicone / helicone

🧊 LLM-Observability for Developers. The open-source platform for logging, monitoring, and debugging.
https://www.helicone.ai
Apache License 2.0
1.5k stars 159 forks source link

Removing a request from cache #1667

Open jumpingtom1 opened 3 months ago

jumpingtom1 commented 3 months ago

This is a problem that I've had several times. I'm running a system of function calls to OpenAI and the responses are JSON. Sometimes, the openai completion doesn't fit the expected function schema, just omitting complete several of the key/values. The returned object is a complete JSON object, so that doesn't throw the error, but as the response is further processed by my app, the lack of certain values makes the app not function properly.

I can diagnose that as the problem by looking at the request logs and finding the response that was incomplete. I would like to be able to manually delete that request from my logs, so that it isn't returned to me as a cached value when I re-run the call. As the system currently exists, the stored request is returned to me from the cache, perpetuating the error.

If its helpful, here is the id of the request I'm referring to: 18f47876-f83b-4cdc-b9ab-f0cb283c006c

Thanks! Helicone is great! I'm looking forward to additional support for the prompts tracking feature.

colegottdank commented 3 months ago

Hi, thank you for the issue and we appreciate your support :)

Would the header Helicone-Cache-Seed solve this for you?

https://docs.helicone.ai/features/advanced-usage/caching#adding-cache-seed