convert_data2vec_audio_original_pytorch_checkpoint_to_pytorch.py works for data2vec 1.0 checkpoint but not data2vec 2.0

Bogsamurai commented 5 months ago

System Info

transformers version: 4.41 Platform: Red Hat Enterprise Linux 8.4 (Ootpa) Python version: 3.10.14 PyTorch version 2.3+cu118

Who can help?

@sanchit-gandhi @SunMarc

Information

[X] The official example scripts
[X] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction

Run convert_data2vec_audio_original_pytorch_checkpoint_to_pytorch.py AttributeError: 'ModuleList' object has no attribute 'mlp'

Expected behavior

Expected behavior is that model weights must be loaded via this function: recursively_load_weights(model, hf_wav2vec, not is_finetuned) but error is:

amyeroberts commented 4 months ago

cc @kamilakesbi

amyeroberts commented 1 month ago

cc @ylacombe

ylacombe commented 1 month ago

Hey @Bogsamurai, correct me if I'm wrong, but I'm pretty sure we don't support data2vec 2.0. There's seems to be some architectural differences between 1.0 and 2.0, according to this. Would you like to work on adding data2vec 2.0 support ?

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

huggingface / transformers