allegroai / clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
https://clear.ml/docs
Apache License 2.0
5.61k stars 651 forks source link

Metrics reporting with OutputModel while in offline mode results in an error #1172

Open GuillaumeRoy33 opened 9 months ago

GuillaumeRoy33 commented 9 months ago

Describe the bug

Calling one of the reporting methods of the OutputModel class after its task has been set offline results in an error. The method crashes with the following error:

File "/home/(...)/.venv/lib/python3.9/site-packages/clearml/backend_interface/metrics/interface.py", line 62, in __init__
    offline_folder = Path(task.get_offline_mode_folder())
AttributeError: 'OutputModel' object has no attribute 'get_offline_mode_folder'

To reproduce

Here is a code bit reproducing this error:

Task.set_offline(offline_mode=True)
task = Task.init(
    project_name="test",
)
output_model = OutputModel(task=Task.current_task())
output_model.report_scalar(title="Some title", series="Some series", value=1.0, iteration=5)

Expected behaviour

I would have expected the OutputModel class to either log the metrics in its local file system or, if this is not possible, raise an error earlier when it was initialized with an offline task.

Environment

ainoam commented 9 months ago

Thanks for reporting @GuillaumeRoy33. We'll take a look.

pollfly commented 4 months ago

Hey @GuillaumeRoy33! Just letting you know that this issue has been resolved in v1.15.0. Let us know if there are any issues :)