Open ValenCassa opened 4 months ago
I don't know if this is related but even the normal open models for streamObject return NaN for usage values.
@sheldonj for OpenAI it should work. In case you use createOpenAI
, you need to set compatibility
to strict
: https://sdk.vercel.ai/providers/ai-sdk-providers/openai#provider-instance (this was necessary to prevent breaking changes with OpenAI-"compatible" providers).
@sheldonj for OpenAI it should work. In case you use
createOpenAI
, you need to setcompatibility
tostrict
: https://sdk.vercel.ai/providers/ai-sdk-providers/openai#provider-instance (this was necessary to prevent breaking changes with OpenAI-"compatible" providers).
Confirmed that using strict mode returns usage! Thank you
But to the OP, it doesn't work for Groq.
Follow here: https://github.com/vercel/ai/discussions/2290
Description
Seems like Groq models do not follow the OpenAI spec for usage. They return the
usage
object under ax-groq
key in the last chunk:Code example
Additional context
No response