sozercan / aikit

🏗️ Fine-tune, build, and deploy open-source LLMs easily!
https://sozercan.github.io/aikit/
MIT License
384 stars 28 forks source link

[REQ] auto detect runtime for an inference container #181

Closed sozercan closed 5 months ago

sozercan commented 7 months ago

What kind of request is this?

Improvement of existing experience

What is your request or suggestion?

https://github.com/mudler/LocalAI/issues/1888

Are you willing to submit PRs to contribute to this feature request?

sozercan commented 7 months ago

docker already has this functionality built-in with platforms (such as linux/amd64/v2, linux/amd64/v3, etc) https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels

This doesn't exist for gpus

sozercan commented 5 months ago

fixed with #225