ya-runtime-ai as library for frameworks

[nieznanysprawiciel]

Why:

• We would like to be able to add new frameworks without creating new runtime build

What:

• ya-runtime-ai should become library
• Frameworks will use library and specialize to handle each runtime
• We won't have any common binary. We will distribute separate binaries for each runtime

Alternative:

• Create API between ya-runtime-ai and python
  • This way we avoid making Rust competence necessary to integrate new frameworks

[pwalski]

Each implementation of ya-runtime-ai library will have:

• framework specific cmdline parameters (so it will be easy to have them configured in runtime descriptor (instead of using some additional config files))
• framework specific offer-template/test cmdline args implementation (e.g. test if framework can start its API server)
• framework process startup (adding some framework specific cmdline args (model, chat template, etc.))
• framework process startup monitoring (e.g. monitoring logs to verify if model was loaded properly)
• healthcheck monitoring (pinging some framework specific endpoint)
• framework specific shutdown implementation (e.g. calling a /kill endpoint, doing some cleanup)