Best way to record data

[aodhan-domhnaill]

I am looking to record input and output to the vLLM. I could put an HTTP proxy in front and capture the traffic, or modify your handler.

Rather than make changes to the code, I was wondering if you might have a better way to do this?

[alpayariyak]

We currently do not have a way to capture token usage across all workers, but it is a future item we will implement