Since token.js currently integrates with every LLM provider using their javascript SDK, the package size is much larger than necessary. This can impact the performance of backend services using token.js in serverless functions.
Solution
Refactor LLM provider integrations to work directly with each providers API instead of using the SDK.
Comments
It's recommended to do this one at a time. The following providers still need to be refactored to remove their SDKs.
[ ] OpenAI
[ ] AWS Bedrock
[ ] Gemini
[x] AI21
[ ] Anthropic
[ ] Cohere
[ ] Mistral
Note that the OpenAI SDK is also used to integrate Perplexity and Groq, so they will need to be refactored as well when removing that SDK.
Problem
Since
token.js
currently integrates with every LLM provider using their javascript SDK, the package size is much larger than necessary. This can impact the performance of backend services usingtoken.js
in serverless functions.Solution
Refactor LLM provider integrations to work directly with each providers API instead of using the SDK.
Comments
It's recommended to do this one at a time. The following providers still need to be refactored to remove their SDKs.
Note that the OpenAI SDK is also used to integrate Perplexity and Groq, so they will need to be refactored as well when removing that SDK.