allegroai / clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
https://clear.ml/docs
Apache License 2.0
5.69k stars 655 forks source link

Task.set_project not working #1190

Closed CourchesneA closed 10 months ago

CourchesneA commented 10 months ago

Describe the bug

I am using pytorch lightning, and I use the CLI module. Since the argument parsing happens at the same time as the training, I don't have access to all the information when I call clearml.Task.init(). So I set info manually after training + parsing using mytask.set_project() and mytask.set_name(). The end result is that only the name was set, but the project was not modified.

To reproduce

clearml_task = Task.init(
    project_name="default_project", task_name=f"default_task",
    output_uri="s3://my_uri"
)

cli = pl.CLI(MyModel)  # parse the config file using jsonargparse and train

clearml_task.set_project(cli.config.fit.myproject)        # Doesn't work
clearml_task.move_to_project(cli.config.fit.myproject)    # Also doesn't work
clearml_task.set_name(cli.config.fit.mytaskname)     # This one work

Expected behaviour

Expected: clearml has task "mytaskname" in project "myproject"

Actual: clearml has task "mytaskname" in project "default_project"

Environment

ainoam commented 10 months ago

@CourchesneA Do note that set_project() and move_to_project take either project ID or project name, do you should name your parameter (which seems to be what you did here).

Does this help?

CourchesneA commented 10 months ago

That's what I was missing, thanks !