When I fine-tune from Meta on a new dataset should I pass `--fine`

[Ca-ressemble-a-du-fake]

Hi,

I want to make sure I am using Toucan properly. So far I've never used the --fine argument (false by default) although I am willing to fine tune the Meta model on my data. So I am using python3 run_training_pipeline.py fine_ME_FR --gpu_id 0 --wandb 2>&1 with fine_ME_FR being a copy of finetuning_example.py as described in the guide.

But while looking at the code I see fine_tune: whether to load everything from a checkpoint, or only the model parameters so I am not sure to use it correctly. What's the difference between passing True or False as value for fine argument ? Should I pass it True when fine tuning ?

Thanks in advance for shedding light on this :smiley:

[Flux9665]

If you start the train script with the --finetune flag, the optimizer and scheduler from the state dict will be ignored, only the model arameters will be loaded. So this is what you have to do if you want to finetune from another checkpoint (so in your case, it should be done).

In the finetuning_example script, it is True by default unless you are providing a previous checkpoint manually (if you e.g. want to continue a crashed run). So if your script is based off of the finetuning_example, you don't need to set the --finetune flag, because this script is made specifically for finetuning. For other pipelines, where you might not always want to finetune a model, but train one from scratch or finetune, you have to control this with the flag during startup.

[Ca-ressemble-a-du-fake]

So it looks like I am using it correctly since I use scripts based off finetuning_example. Thanks for the clarification !