search

snorkel-team / snorkel

A system for quickly generating training data with weak supervision
https://snorkel.org
Apache License 2.0
5.81k stars 857 forks source link

is it possible to use snorkel in ASR (Speech to Text)? #1563

Closed iou2much closed 4 years ago

iou2much commented 4 years ago

Is your feature request related to a problem? Please describe.

I wonder is it possible to use snorkel for ASR task? As I have quite a lot of speech-text pair data, but they're not accurate. Its CER is more than 10%.

Describe the solution you'd like

Someone can show me how to deal with a similar situation? Maybe snorkel used for seq2seq problem.

bhancock8 commented 4 years ago

Great question! In general, you'll find more participation on questions like this in the Snorkel forum (https://spectrum.chat/snorkel). I'll give a quick answer here, and then if you have follow-up questions, let's continue this discussion on the forum!

Snorkel is most easily applied to problems that can be framed as a classification task of some sort; i.e., you're trying to map each input to a particular label in a closed set of options (generally with relatively low cardinality, so not in the thousands). And its primary role is helping you to label large quantities of unlabeled data that you have. So for a seq2seq problem, if you have proposed pairs of speech and text, you could very well use Snorkel to help label which of these you believe are correct or incorrect, and use those results to train your seq2seq model. We would not typically expect to see Snorkel being used to actually generate proposed text for given speech inputs.

iou2much commented 4 years ago

Thank you for your reply. Yes. We use snorkel for several classification cases before, and it's fabulous, and we also have tasks like seq2seq, which seems hard to apply snorkel solution. But I was thinking, since snorkel could be used for NER task, maybe I can use it for sentence/grammar correction or something like that. So I can correct my weak ASR label somehow