search

Portkey-AI / gateway

A Blazing Fast AI Gateway with integrated Guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
https://portkey.ai/features/ai-gateway
MIT License
5.87k stars 403 forks source link

[Provider] Support for Nvidia NeMo #495

Open vrushankportkey opened 1 month ago

narengogi commented 3 weeks ago

Models hosted using Nvidia's NeMo servers expose Nvidia's Triton inference API's, we have a PR for that already (currently only for text completions, not chat completions) https://github.com/Portkey-AI/gateway/pull/445

image

https://docs.nvidia.com/nemo-framework/user-guide/latest/overview.html