Is there a plan to support the ai gateway?

easegress-io / easegress

A Cloud Native traffic orchestration system

https://megaease.com/easegress/

Apache License 2.0

5.78k stars 501 forks source link

Open sodaRyCN opened 5 months ago

xxx7xxxx commented 5 months ago

How do you define AI part in terms of Gateway? If it suits the current market, sure why not.

sodaRyCN commented 4 months ago

the features such as

tpm/rpm limits
keyword filter
token metrics
mulit LLM protocol(sse and others stream protocol)
... There are currently some existing gateways that support AI such as Kong, LiteLLM...