coqui-ai / Trainer

🐸 - A general purpose model trainer, as flexible as it gets
196 stars 116 forks source link

[Bug] Trainer saves the given config to ClearML rather than the continued config #55

Open erogol opened 2 years ago

erogol commented 2 years ago

Describe the bug

Trainer saves the config that is given at the beginning of the training rather than the one restored after continuing a training.

To Reproduce

Continue one of your training using python script and give a modified config.json

The trainer will use the restored config but save the modified config.

Expected behavior

Save the config as exactly used.

Logs

No response

Environment

{
    "CUDA": {
        "GPU": [
            ""
        ],
        "available": true,
        "version": "11.3"
    },
    "Packages": {
        "PyTorch_debug": false,
        "PyTorch_version": "1.11.0",
        "Trainer": "v0.0.12",
        "numpy": "1.21.6"
    },
    "System": {
        "OS": "Linux",
        "architecture": [
            "64bit",
            "ELF"
        ],
        "processor": "x86_64",
        "python": "3.9.5",
        "version": "#119-Ubuntu SMP Mon Mar 7 18:49:24 UTC 2022"
    }
}

Additional context

No response