search

InftyAI / llmaz

☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
Apache License 2.0
31 stars 10 forks source link

An an example for multi-host inference with Service #61

Open kerthcet opened 3 months ago

kerthcet commented 3 months ago

What would you like to be added:

Right now we don't support multi-host in playground, but this is supported in Service, we should provide an example.

Why is this needed:

Completion requirements:

This enhancement requires the following artifacts:

The artifacts should be linked in subsequent comments.

kerthcet commented 3 months ago

/kind cleanup /milestone v0.1.0