Bump transformers from 4.24.0 to 4.25.1

[dependabot[bot]]

Bumps transformers from 4.24.0 to 4.25.1.

Release notes

Sourced from transformers's releases.

PyTorch 2.0 support, Audio Spectogram Transformer, Jukebox, Switch Transformers and more

PyTorch 2.0 stack support

We are very excited by the newly announced PyTorch 2.0 stack. You can enable torch.compile on any of our models, and get support with the Trainer (and in all our PyTorch examples) by using the torchdynamo training argument. For instance, just add --torchdynamo inductor when launching those examples from the command line.

This API is still experimental and may be subject to changes as the PyTorch 2.0 stack matures.

Note that to get the best performance, we recommend:

• using an Ampere GPU (or more recent)
• sticking to fixed shaped for now (so use --pad_to_max_length in our examples)

• Repurpose torchdynamo training args towards torch._dynamo by @​sgugger in #20498

Audio Spectrogram Transformer

The Audio Spectrogram Transformer model was proposed in AST: Audio Spectrogram Transformer by Yuan Gong, Yu-An Chung, James Glass. The Audio Spectrogram Transformer applies a Vision Transformer to audio, by turning audio into an image (spectrogram). The model obtains state-of-the-art results for audio classification.

• Add Audio Spectogram Transformer by @​NielsRogge in #19981

Jukebox

The Jukebox model was proposed in Jukebox: A generative model for music by Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, Ilya Sutskever. It introduces a generative music model which can produce minute long samples that can be conditionned on an artist, genres and lyrics.

• Add Jukebox model (replaces #16875) by @​ArthurZucker in #17826

Switch Transformers

The SwitchTransformers model was proposed in Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity by William Fedus, Barret Zoph, Noam Shazeer.

It is the first MoE model supported in transformers, with the largest checkpoint currently available currently containing 1T parameters.

• Add Switch transformers by @​younesbelkada and @​ArthurZucker in #19323

RocBert

The RoCBert model was proposed in RoCBert: Robust Chinese Bert with Multimodal Contrastive Pretraining by HuiSu, WeiweiShi, XiaoyuShen, XiaoZhou, TuoJi, JiaruiFang, JieZhou. It’s a pretrained Chinese language model that is robust under various forms of adversarial attacks.

• Add RocBert by @​sww9370 in #20013

CLIPSeg

The CLIPSeg model was proposed in Image Segmentation Using Text and Image Prompts by Timo Lüddecke and Alexander Ecker. CLIPSeg adds a minimal decoder on top of a frozen CLIP model for zero- and one-shot image segmentation.

• Add CLIPSeg by @​NielsRogge in #20066

NAT and DiNAT

NAT

NAT was proposed in Neighborhood Attention Transformer by Ali Hassani, Steven Walton, Jiachen Li, Shen Li, and Humphrey Shi.

... (truncated)

Commits

• 31d452c Release v4.25.1
• 7378726 Release: v4.25.0
• e342ac7 Add some warning for Dynamo and enable TF32 when it's set (#20515)
• 68cfffc Fix Data2VecTextForCasualLM example code documentation (#20510)
• dd6fb13 Add natten for CI (#20511)
• afb6674 Update AutomaticSpeechRecognitionPipeline doc example (#20512)
• 04c653a Fix style
• 7217640 Add Chinese-CLIP implementation (#20368)
• 396a6a2 Fix minimum version for device_map (#20489)
• 08b4621 Repurpose torchdynamo training args towards torch._dynamo (#20498)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[dependabot[bot]]

Looks like transformers is up-to-date now, so this is no longer needed.