InftyAI / llmaz

☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
Apache License 2.0
30 stars 10 forks source link

Support scaling with Spot instances for cost saving #106

Open kerthcet opened 2 months ago

kerthcet commented 2 months ago

What would you like to be added:

See https://github.com/spotinst as an example, which means we should support multi cloud providers.

Why is this needed:

Cost saving for users.

Completion requirements:

This enhancement requires the following artifacts:

The artifacts should be linked in subsequent comments.

kerthcet commented 2 months ago

/kind question