`BarkModel` can't be saved anymore

[IlyasMoutawwakil]

System Info

• transformers version: 4.43.2
• Platform: Linux-5.4.0-166-generic-x86_64-with-glibc2.35
• Python version: 3.10.12
• Huggingface_hub version: 0.24.2
• Safetensors version: 0.4.3
• Accelerate version: 0.33.0
• Accelerate config: not found
• PyTorch version (GPU?): 2.3.1+cpu (False)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using distributed or parallel set-up in script?:

Who can help?

@sanchit-gandhi

Information

• [ ] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

from transformers import AutoModel

bark = AutoModel.from_pretrained("ylacombe/bark-small")
bark.save_pretrained("local_model")

fails with

/usr/local/lib/python3.10/dist-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
  warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")
/usr/local/lib/python3.10/dist-packages/transformers/models/encodec/modeling_encodec.py:120: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  self.register_buffer("padding_total", torch.tensor(kernel_size - stride, dtype=torch.int64), persistent=False)
Traceback (most recent call last):
  File "/home/ilyas/optimum/sanity.py", line 5, in <module>
    bark.save_pretrained("local_model")
  File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 2679, in save_pretrained
    raise RuntimeError(
RuntimeError: The weights trying to be saved contained shared tensors [{'fine_acoustics.input_embeds_layers.1.weight', 'fine_acoustics.lm_heads.0.weight'}, {'fine_acoustics.lm_heads.1.weight', 'fine_acoustics.input_embeds_layers.2.weight'}, {'fine_acoustics.input_embeds_layers.3.weight', 'fine_acoustics.lm_heads.2.weight'}, {'fine_acoustics.lm_heads.3.weight', 'fine_acoustics.input_embeds_layers.4.weight'}, {'fine_acoustics.input_embeds_layers.5.weight', 'fine_acoustics.lm_heads.4.weight'}, {'fine_acoustics.lm_heads.5.weight', 'fine_acoustics.input_embeds_layers.6.weight'}, {'fine_acoustics.lm_heads.6.weight', 'fine_acoustics.input_embeds_layers.7.weight'}] that are mismatching the transformers base configuration. Try saving using `safe_serialization=False` or remove this tensor sharing.

Expected behavior

to save the model locally

[LysandreJik]

cc @ylacombe as well

[ylacombe]

Thanks for opening the issue @IlyasMoutawwakil, I've opened a PR to fix this!