kubeflow / arena

A CLI for Kubeflow.
Apache License 2.0
720 stars 176 forks source link

⚠️ Breaking Changes: Migrate model subcommand to model analyze #1061

Closed ChenYi015 closed 3 months ago

ChenYi015 commented 3 months ago

Currently, the arena model subcommand is used for submitting a model analysis job, with available job types being profile, evaluate, optimize and benchmark. These jobs have nothing to do with model management. To integrate some form of model registry, we first need to migrate the arena model subcommand to somewhere else, like arena model analyze.

Before migration:

$ arena model --help
submit a model analyze job.

Usage:
  arena model [flags]
  arena model [command]

Available Commands:
  benchmark   Submit a model benchmark job
  delete      Delete a model job
  evaluate    Submit a model evaluate job
  get         Get a model job
  list        List all the model jobs
  optimize    Submit a model optimize job, this is a experimental feature
  profile     Submit a model profile job

After migration:

$ arena model analyze --help                    
Submit a model analyze job. (experimental feature)

Usage:
  arena model analyze [flags]
  arena model analyze [command]

Available Commands:
  benchmark   Submit a model benchmark job
  delete      Delete a model analyze job
  evaluate    Submit a model evaluate job
  get         Get a model analyze job
  list        List all model analyze jobs
  optimize    Submit a model optimize job
  profile     Submit a model profile job
Syulin7 commented 3 months ago

Agree, especially considering that this feature is an experimental feature. As stated in https://github.com/kubeflow/arena/issues/1059, we need to support model management in Arena, so I believe it is necessary to migrate the model subcommand to analyze. Perhaps we can complete this in the next release version.

cc @cheyang @gujingit