search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
133.88k stars 26.78k forks source link

Is it possible to support Wav2Vec in ZeroShotClassificationPipeline? #15222

Closed mabu-dev closed 2 years ago

mabu-dev commented 2 years ago

Similar to the very helpful NLI-based zero-shot classification pipeline using a ModelForSequenceClassification, it would be great to have zero-shot on audio data.

Pass wav file(s) with candidate labels to the pipeline and get a prediction. Is this at all on the roadmap?

LysandreJik commented 2 years ago

Maybe of interest to @Narsil and @patrickvonplaten

patrickvonplaten commented 2 years ago

Pretty cool idea! @mabu-dev do you know whether there is a research paper on it? For NLI zero-shot we have this paper: https://huggingface.co/facebook/bart-large-mnli#nli-based-zero-shot-text-classification

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.