search

arakoodev / EdgeChains

EdgeChains.js is Full-Stack GenAI library. Front-end, backend, apis, prompt management, distributed computing. All core prompts & chains are managed declaratively in jsonnet (and not hidden in classes)
https://www.arakoo.ai/
GNU Affero General Public License v3.0
353 stars 72 forks source link

evaluate Triton standard for serving LLM #103

Open sandys opened 1 year ago

sandys commented 1 year ago

https://github.com/triton-inference-server/backend#backends

sandys commented 1 year ago

https://github.com/triton-inference-server/client/tree/main/src/java/src/main/java/triton/client/examples