FEAT: Preservation and auto-restoration of deployed models post-service restart

[onesuper]

The problem

Every time I undergo an upgrade or reinstallation, I find myself having to restart the xinference service. A pain point during this process is the loss of information related to already deployed models. For instance, if I have four different versions of a model deployed, I currently have to restart each of these models post-service restart manually.

What I am thinking

I propose that xinference integrates a feature that allows the service to remember the details of deployed models.

• Remember the Models: Just keep track of all the models I've got running - which ones, their quantities and formats.
• Auto-Comeback: After a restart, just have all my models come back online. No fuss.
• Keep the UIDs: Make sure the model UIDs stay the same. It's just easier that way.

Implementing this feature will significantly enhance the user experience, reduce manual overhead, and ensure seamless continuity in operations even after system upgrades or restarts.

Another idea

If the auto-restart thing is tricky, how about giving me a command that lets me save all my model info into a script? Then, after a restart, I can just run that script and get all my models back online. It's not fully automatic, but it's way better than what I'm doing now.

[wencan]

I think this is very important. In actual production environments, services may need to be restarted for various reasons. We must ensure that the service remains online at all times.

[github-actions[bot]]

This issue is stale because it has been open for 7 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 5 days since being marked as stale.