Finetuning from MTR models was causing issues because of matching weights names. Essentially, pytorch would try to load the same weights even when the output shape had changed. This fix works by loading the state_dict manually, removing the weights for the last layer, and then passing that into the HuggingFace .from_pretrained method
Finetuning from MTR models was causing issues because of matching weights names. Essentially, pytorch would try to load the same weights even when the output shape had changed. This fix works by loading the
state_dictmanually, removing the weights for the last layer, and then passing that into the HuggingFace.from_pretrainedmethod