search

allegroai / clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
https://clear.ml/docs
Apache License 2.0
5.69k stars 655 forks source link

HPO instances appear in the Projects view even if they are pipelines #1014

Open Make42 opened 1 year ago

Make42 commented 1 year ago

If one does a hyperparameter optimization (HPO) from a script, using HyperParameterOptimizer, the tasks created by the HPO (HPO instances) appear in the same project as the original base_task_id project, by default.

If the base task is a simple experiment, i.a., not a pipeline, the base task is visible in the "Projects" view in the Web UI, and so are the created HPO instances.

If the base task is a pipeline, then the base task is in the "Pipelines" view in the Web UI and not in the "Projects" view. All good so far. In this case that the base task is a pipeline, I would expect that the HPO instances are visible in the "Pipelines" view as well, since (as clones from the base task), they are pipelines. However, this is not the case. Instead, these HPO instances are visible in the "Projects" view. This is unexpected for me. There is a (strange) workaround though. If I do not use the default of the argument

:param str spawn_project: If project name is specified, create all optimization Jobs (Tasks) in the specified project instead of the original base_task_id project.

of HyperParameterOptimizer, but instead provide the project name of the original base task pipeline explicitly, then the HPO instances appear in the "Pipelines" view.

Somehow I doubt this is intended behavior. The best solution might be an argument with which the programmer can determine where the HPO instances should appear? By default, I would expect in the same view as the base task.

ainoam commented 1 year ago

Thanks for pointing this out @Make42,

We'll take a look and update on progress.