search

allegroai / clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
https://clear.ml/docs
Apache License 2.0
5.69k stars 655 forks source link

Make the Optuna pruning an informative log message #1061

Open fojoshi opened 1 year ago

fojoshi commented 1 year ago

Proposal Summary

Make the optuna pruning experiment an informative message

Motivation

While executing the hyper-parameter task with the optimizer set to Optuna, it would be terminating experiments based on the pruning algorithm. However, the status shows Aborted but with the message log showing, User aborted: stopping task (3) which is incoherent. It would be great to get a detailed message about why the experiment was aborted.

Related Discussion

If this continues a slack thread, please provide a link to the original slack thread.

ainoam commented 1 year ago

@fojoshi It would be tricky for a task to know the reason why it had been stopped: That information is available to the logic that controls execution, in the ClearML case - The optimizer task.

When you go over your optimizer's log - Do you not see which tasks were pruned?