The OpenAI server was always returning "usage":{"prompt_tokens":2,"total_tokens":4,"completion_tokens":2} since it was taking the length of the token_ids dictionary and not one of its members
{"id":"cmpl-10b937d72d2b4cf382da0c6e54241814","object":"chat.completion","created":1703173842,"model":"hf:mgoin/TinyStories-1M-ds","choices":[{"message":{"role":"assistant","content":" You can make a big house for the house and the house. You can make"},"finish_reason":"length"}],"usage":{"prompt_tokens":2,"total_tokens":4,"completion_tokens":2}}
After:
{"id":"cmpl-85104f94d3294b88b2aae34095cf204d","object":"chat.completion","created":1703173678,"model":"hf:mgoin/TinyStories-1M-ds","choices":[{"message":{"role":"assistant","content":" You can make a big house for the house and the house. You can make"},"finish_reason":"length"}],"usage":{"prompt_tokens":6,"total_tokens":22,"completion_tokens":16}}
The OpenAI server was always returning
"usage":{"prompt_tokens":2,"total_tokens":4,"completion_tokens":2}
since it was taking the length of the token_ids dictionary and not one of its membersTest
Server command:
Client command:
Before:
After: