Tharsanan1
commented
4 days ago Feature added with : https://github.com/wso2/apk/pull/2446
Closed Tharsanan1 closed 4 days ago
Tharsanan1
commented
4 days ago Feature added with : https://github.com/wso2/apk/pull/2446
Problem
We are introducing AI APIs to apk. In the AI usecases its a common usecase to limit the requests based on the token counts(prompt, completion, total).
Solution
Add a feature to support subscription scoped ratelimiting AI requests based on the token count.
Affected Component
None
Version
No response
Implementation
No response
Related Issues
No response
Suggested Labels
No response