Open ZeguanXiao opened 1 year ago
Also, it seems EncoderDecoderModelAdaptersMixin.iter_layers
should count decoder layer_id starting with len(self.encoder.layers)) like this?
def iter_layers(self) -> Iterable[Tuple[int, nn.Module]]:
for i, layer in self.encoder.iter_layers():
yield i, layer
encoder_layer_n = len(self.encoder.encoder.layer)
for i, layer in self.decoder.iter_layers():
yield i + encoder_layer_n, layer
Hey @ZeguanXiao, I see why this is unexpected behavior. Unfortunately it is not as easy as changing the iter_layer
indices. I will look into this.
@hSterz My current workaround is setting model.decoder.base_model.config.adapters = model.encoder.base_model.config.adapters
and changing theiter_layer
. It seems to work fine.
This issue has been automatically marked as stale because it has been without activity for 90 days. This issue will be closed in 14 days unless you comment or remove the stale label.
Environment info
adapter-transformers
version: 3.1.0Information
Model I am using (Bert, XLNet ...): EncoderDecoderModel
Language I am using the model on (English, Chinese ...): English
Adapter setup I am using (if any): AdapterConfig
The problem arises when using:
The tasks I am working on is:
To reproduce
When not leaving out layers, it's okay.
When trying to leave out all encoder layers, not any adapter is added.
When only leaving out the first 6 layers of the encoder, we see adapters are only added to the encoder, leaving the decoder.
Expected behavior
The EncoderDecoderModel class work like BART-like models.