ml-explore / mlx-examples

Examples in the MLX framework
MIT License
5.5k stars 791 forks source link

Server loads the model on demand from the request #851

Closed angeloskath closed 5 days ago

angeloskath commented 6 days ago

As title. Basically moves the model loading logic into the ModelProvider. The POST handler passes the requested_model to the model provider to load that model instead of always using the default one.

It should be completely backwards compatible even wrt to when the default model will be loaded.