How to use language model?

[wfredc1996]

i am developing a realtime pvp with speech recognition. I tried this plugin. i like it, but now can use keyword and grammer only, how about lm? And will you implement the training process into ue4?

[shanecolb]

> but now can use keyword and grammer only, how about lm?

Language Models are generally for free form text detection. I didn't see much need for this, and as such omitted it. Free form text detection has pretty poor accuracy (in Pocketsphinx) from my experimentation. It'd be pretty easy to add in, and I could probably send you the necessary changes that would be required to the source code, if you wanted to try it out? Can probably send you the code changes over the weekend.

> And will you implement the training process into ue4?

In the long-term future, I would like to add in model adaption into the plugin. However, I have many more pressing things to work on (nothing to do with this plugin). Feel free to clone the repo, and make changes to your own copy however you wish. Then let me know, and if worthwhile, I can merge changes into the trunk.

[lacvapps]

@shanecolb This is something I would be interested in if you were willing to send me the changes. I am working on a chat bot application in unreal vr. Thank you.

[shanecolb]

Hi Everyone, Sorry about the delay, I have been rather busy. I have added language model support to a demo project. Please download the following demo project.

https://drive.google.com/open?id=0BxR5qe2wdwSLWmdBNlpBTTN5YkU

Have a look at LanguageModel_bp blueprint.

Note: Loading language models takes a few seconds, so it is slower to initialise than the other methods. When the Output log says 'Listening', that is when it is truly active.

I tested with the first 3 lines of X-files S1E01...because...I love the show, and just felt like it!

"They put the time of death between eight and twelve hours ago." "No visible cause, no sign of battery or sexuaI assault." "All we have is this."

Here's the result. Pretty decent, and the faults were close phonetic matches. I think i`ll definitely look at adding this into a future release.

LogBlueprintUserMessages: [LanguageModel_bp_C_0] they LogBlueprintUserMessages: [LanguageModel_bp_C_0] put LogBlueprintUserMessages: [LanguageModel_bp_C_0] the LogBlueprintUserMessages: [LanguageModel_bp_C_0] time LogBlueprintUserMessages: [LanguageModel_bp_C_0] of LogBlueprintUserMessages: [LanguageModel_bp_C_0] death LogBlueprintUserMessages: [LanguageModel_bp_C_0] between LogBlueprintUserMessages: [LanguageModel_bp_C_0] a LogBlueprintUserMessages: [LanguageModel_bp_C_0] twelve LogBlueprintUserMessages: [LanguageModel_bp_C_0] hours LogBlueprintUserMessages: [LanguageModel_bp_C_0] ago

LogBlueprintUserMessages: [LanguageModel_bp_C_0] the LogBlueprintUserMessages: [LanguageModel_bp_C_0] visible LogBlueprintUserMessages: [LanguageModel_bp_C_0] cause LogBlueprintUserMessages: [LanguageModel_bp_C_0] those LogBlueprintUserMessages: [LanguageModel_bp_C_0] honest LogBlueprintUserMessages: [LanguageModel_bp_C_0] battery LogBlueprintUserMessages: [LanguageModel_bp_C_0] or LogBlueprintUserMessages: [LanguageModel_bp_C_0] sexual LogBlueprintUserMessages: [LanguageModel_bp_C_0] assault

LogBlueprintUserMessages: [LanguageModel_bp_C_0] all LogBlueprintUserMessages: [LanguageModel_bp_C_0] we LogBlueprintUserMessages: [LanguageModel_bp_C_0] have LogBlueprintUserMessages: [LanguageModel_bp_C_0] is LogBlueprintUserMessages: [LanguageModel_bp_C_0] this

Let me know if there's any serious problems with getting it to work. Cheers. Shane

[lacvapps]

Awesome thank you man! Good work.

[TheYojimbo]

Hello Shane, first, thank you very much for your work ! This is awesome and seems to work pretty well.

Now, I'm trying to use it for my project where I need to have a text input in VR, but virtual keyboards are not really user friendly. So I downloaded your demo project, didn't found any langage model integration, and found this thread, downloaded the test project and it seems to work pretty well !

Now, my problem is, I'm french and need french recognition. So I tried :

• set "French" in the Init Node
• copying the "fr" folder from the demo project to the test project.
• put "fr" as language model in the "Enable Language Model Mode" Node
• put the "fr.lm" file in an "language_models" folder

But it didn't work. I don't get anything printed from the "EventsWordSpoken" node. I think it's because I'm missing some files in the "fr" folder, but can't figure out how to find them...

Thanks in advance for your help, and again, excellent work, you rock !

Edit : Ok so I tried downloading the language model from another source, and it seems to work (well kinda, but I guess I won't have much better...) So I'll try working with that ! Thanks a lot :)