search

allegroai / clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
https://clear.ml/docs
Apache License 2.0
5.61k stars 651 forks source link

Task.connect() support for ml_collections.ConfigDict #1035

Open mhtrinh opened 1 year ago

mhtrinh commented 1 year ago

We are using ml_collections.ConfigDict to store our training configurations. ClearML's Task seems to not be able to "parse" ml_collections.ConfigDict object. Example: 

from ml_collections import ConfigDict
from clearml import Task

task = Task.init(project_name='test', task_name='test-config-connect',reuse_last_task_id=False)
cfg = ConfigDict()
cfg.param1 = 5
cfg.epoch=10
print(cfg)
task.connect(cfg,name="myconfig")

Once the task completed, in the UI, in configuration, there is no myconfig

I can use task.connect(cfg.to_dict(),name="myconfig") in order to get the Configuration poppulated. But then if I clone the task and change configuration to run with different parameter, those are not populated to the task, which guess because the connect is happening to a returned dict rather than the configuration object itself.

ainoam commented 1 year ago

Thanks for suggesting @mhtrinh :)

We'll look at addressing this in a near release.