Open LuisFX opened 3 months ago
API call result object would include the token utilization For accounting reasons, it would be useful to get the token utilization for each call. Akin to what openai includes along other metadata.
I do agree, It would be great to get in in all models. I would use it in VertexAI model.
This is must have feature, it is rly hard to use flowine in production without option to know how much user spend @HenryHengZJ
API call result object would include the token utilization For accounting reasons, it would be useful to get the token utilization for each call. Akin to what openai includes along other metadata.