Xmod model has not module 'roberta'

[WLucky]

System Info

latest version

Who can help?

No response

Information

• [X] The official example scripts
• [ ] My own modified scripts

Tasks

• [x] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

path: src/transformers/models/xmod/modeling_xmod.py

function def freeze_embeddings_and_language_adapters(self)

when i call this function, the error rasie: model no module roberta a similar error raise when load my funtinue xmod checkpoint

and i modify as this (delete roberta), it seem run ok, but I cant certain its really right:

def freeze_embeddings_and_language_adapters(self):
        """
        Freeze the embeddings and language adapters of the model. Usually, this is applied before the model is
        fine-tuned on a downstream task.
        """
        logger.info("Freezing embeddings")
        for parameter in self.embeddings.parameters():
            parameter.requires_grad = False
        logger.info("Freezing adapters")
        for layer in self.encoder.layer:
            if layer.output.adapter_layer_norm is not None:
                for parameter in layer.output.adapter_layer_norm.parameters():
                    parameter.requires_grad = False
            for parameter in layer.output.adapter_modules.parameters():
                parameter.requires_grad = False

Expected behavior

call freeze_embeddings_and_language_adapters with no error raise

[LysandreJik]

Hey @WLucky, do you mind sharing the code snippet that got you this error? As well as the full error, if you can

Thanks!

[WLucky]

Hey @WLucky, do you mind sharing the code snippet that got you this error? As well as the full error, if you can

Thanks!

The reproduce is simple and as follows:

model = XmodModel.from_pretrained(....)
model.set_default_language("en_XX")
model.freeze_embeddings_and_language_adapters()

Then the error will raise in last line. such as not find 'roberta'.

I cant reproduce the error as I already modify the code. I suspect that XmodModel has no roberta module.