GigaAM: the family of open-source acoustic models for speech processing

Table of contents

• GigaAM
• GigaAM for Speech Recognition
  • GigaAM-CTC
  • GigaAM-RNNT
• GigaAM-Emo
• License
• Links

GigaAM

GigaAM (Giga Acoustic Model) is a Conformer-based wav2vec2 foundational model (around 240M parameters). We trained GigaAM on nearly 50 thousand hours of diversified speech audio in the Russian language.

Resources:

• Model weights
• Encoder config
• Colab example
• Docker example

GigaAM for Speech Recognition

We fine-tuned the GigaAM encoder for Speech Recognition with two different decoders:

• GigaAM-CTC was fine-tunined with Connectionist Temporal Classification and character-based tokenizer.
• GigaAM-RNNT was fine-tuned with RNN Transducer loss and subword tokenizer.

Both models were trained using the NeMo toolkit on publicly available Russian labeled data:

Resources:

• GigaAM-CTC:

  • Model weights
  • Model config
  • Colab example
  • Docker example

• GigaAM-RNNT:

  • Model weights
  • Model config
  • Colab example
  • Docker examples

The following table summarizes the performance of different models in terms of Word Error Rate on open Russian datasets:

GigaAM-Emo

GigaAM-Emo is an acoustic model for Emotion Recognition. We fine-tuned the GigaAM Encoder on the Dusha dataset.

Resources:

• Model weights
• Model config
• Colab example
• Docker example

The following table summarizes the performance of different models on the Dusha dataset:

Links

• [habr] GigaAM: класс открытых моделей для обработки звучащей речи
• [youtube] GigaAM: Семейство акустических моделей для русского языка
• [youtube] Speech-only Pre-training: обучение универсального аудиоэнкодера