Closed mkozakov closed 2 years ago
The API response of tokenize now includes a token_strings field. This PR adds it to the SDK.
token_strings
Example:
const response = await cohere.tokenize({ text: 'hello world' }); console.log(response.body["tokens"]) // [33555, 1114] console.log(response.body["token_strings"]) // ["hello", " world"]
The API response of tokenize now includes a
token_strings
field. This PR adds it to the SDK.Example: