search

allegroai / clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
https://clear.ml/docs
Apache License 2.0
5.42k stars 643 forks source link

How to use Omegaconf without Hydra? #1277

Open konstantinator opened 4 weeks ago

konstantinator commented 4 weeks ago

Hi!

I use Omegaconf in my project as a hyperparameter storage solution. Is there a way to track Omegaconf without Hydra, but in a format similar to Hydra's (i.e., to have the Omegaconf object visible in the UI as a YAML structure) and to be able to modify the YAML file in the UI and run the experiment with updated parameters?

If it's impossible without Hydra, is there a way to set the Hydra/allow_omegaconf_edit parameter to always be True? (so that it's always possible to modify the parameters of a cloned experiment in the UI, without needing to manually switch it on a separate tab)

eugen-ajechiloae-clearml commented 3 weeks ago

Hi @konstantinator ! The way clearml connects the OmegaConf config is: it dumps the object as a yaml using OmegaConf.to_yaml then it sets this using Task.set_configuration_object(config_text=OmegaConf.to_yaml(config)) then fetches this using Task.get_configuration_object. You could also use Task.connect_configuration which will set the configuration while running locally and return the configuration (which may be edited from the UI) while running remotely. Hydra/allow_omegaconf_edit can't be currently set to default to True.