search

ARIA-VALUSPA / AVP

This is the ARIA-VALUSPA Platform, or AVP for short. Use this platform to build your own Virtual Humans with audio-visual input and output, language models for English, French, and German, emotional understanding, and many more. This work was funded by European Union Horizon 2020 research and innovation programme, grant agreement No 645378.
https://aria-agent.eu
32 stars 16 forks source link

kaldi command #13

Closed jonlnichols closed 6 years ago

jonlnichols commented 6 years ago

hi, it looks like your kaldi ASR commands ( online2-audio-nnet2-latgen-faster and online2-audio-nnet2-latgen-faster-gpu) are custom kaldi commands.

i wanted to do something similar, but with nnet3 chain models instead of nnet2 models. are you planning on adding support for nnet3 models or if not can you share the code you used to make your commands, so i can try to modify it for nnet3 models.

thanks

mvalstar commented 6 years ago

Hi Jon,

This would be one for Eduardo Coutinho to answer - if possible. I don't think we'll be changing much about the ASR I think, as most of the people working on that have left now. But perhaps Eduardo can share the code or explain how they made them?

Michel

bpotard commented 6 years ago

Hi, I actually wrote most of the code of the ASR server, and I provided it under an Apache license to ICL/Passau. I am personally happy to release it as is - it is a fairly straightforward adaptation of the server code in nnet/online to handle nnet2/online2 "mixed" neural net/fst structures. I believe it should be able to handle nnet3 neural network once re-compiled. I could try to have it merged into kaldi - but that would probably take a while - or we could add the source files "as is" under the same Apache license that kaldi uses, with a script to build the server binary.

@eadward Eduardo, are you and Amr happy with that? From what I can see, I am the only person from ARIA credited in the latest source files, but I know that at least Amr had done some modifications to the server code. He does not appear to have an account on github though. If you prefer, we can also release this under a GPL license (but then that will not be included in kaldi).

bpotard commented 6 years ago

I have now added the source code of the ASR server, as well as a script to build binaries, in a subfolder called ASR/src in the private ARIA repo. I took the opportunity to add an executable capable of using nnet3 models. If everyone is happy with it, it can be made public.

eadward commented 6 years ago

Thanks, Blaise! I think it is perfectly fine to release it.

bpotard commented 6 years ago

Ok, the sources of the ASR server have now been added to the AVP git at ASR/src. @jonlnichols there is the source code for an executable supposed to support nnet3 acoustic models, it builds but it has never been tested!