search

open-webui / pipelines

Pipelines: Versatile, UI-Agnostic OpenAI-Compatible Plugin Framework
MIT License
1.01k stars 319 forks source link

Expose prompt/completion tokens to filter pipelines #250

Closed marcklingen closed 2 months ago

marcklingen commented 2 months ago

Is your feature request related to a problem? Please describe.

I'd like to improve the integration (filter pipeline) with Langfuse (github). Currently only inputs/outputs and some metadata are logged to langfuse. As openwebui is already tracking token counts, I'd like to pipe it also to Langfuse for further analysis.

Token counts I saw in openwebui CleanShot 2024-09-05 at 15 23 36

Describe the solution you'd like Expose prompt/completion token counts in the body passed to filter_pipelines.

Describe alternatives you've considered None, capturing token counts at the source is the best solution to get to accurate metrics.

Additional context

marcklingen commented 2 months ago

@tjbck what do you think about this change? is this a simple addition? I think this would make many of the integrations much more useful as token counts are especially interesting to monitor in larger organizations

tjbck commented 2 months ago

I believe this is already exposed if I'm not mistaken, I currently don't have the capacity to take a look at pipeline related aspects atm unfortunately, so PR welcome!

marcklingen commented 2 months ago

I believe this is already exposed if I'm not mistaken, I currently don't have the capacity to take a look at pipeline related aspects atm unfortunately, so PR welcome!

thanks for the response, I'll have a look

marcklingen commented 2 months ago

Thanks again. Here's the updated pipeline (pr): https://github.com/open-webui/pipelines/pull/263

closing this now