Support for torch.nn.Transformer Class?

mravanelli / pytorch-kaldi

pytorch-kaldi is a project for developing state-of-the-art DNN/RNN hybrid speech recognition systems. The DNN part is managed by pytorch, while feature extraction, label computation, and decoding are performed with the kaldi toolkit.

2.37k stars 446 forks source link

Support for torch.nn.Transformer Class? #245

Open niticon opened 4 years ago

niticon commented 4 years ago

I was hoping to compare performance of Transformer models and non-Transformer models. I'm essentially trying to reproduce BERT which is possible according to the PyTorch documentation. Or am I better off just adding the model class to the neural_nets.py library?

TParcollet commented 3 years ago

Hi, sorry for the late reply, we are very very busy with SpeechBrain and PyTorch-Kaldi won't be maintained anymore very soon. You can easily wrap the pytorch transformers to a model in neural_nets.py.