search

machine-learning-exchange / katalog

MLX Katalog is a project to hold the default content samples to bootstrap Machine Learning eXchange.
Apache License 2.0
12 stars 13 forks source link

Add `container_port` to model YAML #75

Closed ckadner closed 2 years ago

ckadner commented 2 years ago

Currently, when models are served via the "Launch" tab of the MLX UI, the container port is fixed and assumed to be 5000. This works for the containerized MAX models, all of which serve inferencing requests on port 5000. However any other models to be registered in MLX that are not listening on port 5000 would have to be rebuild before they can be registered and deployed in MLX.

This PR introduces a new optional field to the model YAML to allow users to specify the port on which the containerized model listens to inferencing requests.

There is PR #371 on the mlx repo to make use of the container_port parameter from the model YAML in the model-config component that is used in the model deployment pipeline in MLX.

/cc @Tomcli @rafvasq

mlx-bot-app[bot] commented 2 years ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ckadner, Tomcli

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/machine-learning-exchange/katalog/blob/main/OWNERS)~~ [Tomcli,ckadner] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment