Can't extract numbers from intents.

I've tried everything I could think of and have not been able to extract a number from an intent sent by the JovoWebClient lib. Meaning that this.$inputs is always empty —i.e. {}.

My intent looks like this in models/en-US.json:

{
    "name": "LoremIpsumResolve",
    "phrases": [
        "{wordCount} words",
        "{wordCount} lorem words",
        "{wordCount} lorem ipsum words",
        "{wordCount} latin words",
        "say {wordCount} words",
        "say {wordCount} words in latin",
        "speak {wordCount} words in latin",
        "say {wordCount} latin words",
        "speak {wordCount} latin words",
        "speak {wordCount} words of lorem ipsum",
        "give me {wordCount} words in latin"
    ],
    "inputs": [
        {
            "name": "wordCount",
            "type": {
                "alexa": "AMAZON.NUMBER",
                "googleAssistant": "actions.type.Number",
                "nlpjs": "number"
            }
        }
    ]
}

It works perfectly fine with Alexa or Google Assistant, but won't work when the request comes from a client web application. I've also tried "nlpjs": "Number", "nlpjs": "integer" and "nlpjs": "Integer".

I noticed that $request coming from Alexa and Google assistant bears "type": "IntentRequest", but intents coming from the JovoWebClient lib bear "type": "TEXT", so then I tried sending the request as a RequestType.Intent from the web client:

client.createRequest({
    //type: RequestType.Text,
    type: RequestType.Intent,
    body: { text }
}).send()

This resulted in the $request bearing "type": "INTENT", but that didn't do the trick either.

I also noticed that this.$nlu carries NLP.js entities (https://github.com/axa-group/nlp.js/blob/master/docs/v3/builtin-entity-extraction.md#number-extraction) but there's no resolution.

Just to clarify, the console output I'm getting while trying this out is below.

Notice how I start by saying say 5 words of lorem ipsum and it starts off by getting it wrong, sending me to the intent handler where I ask How many words do you need?, as if I had not specified the number.

That aside, I reply with 5 words, and then it gets it right and goes to the intent handler I meant for this, but printing out $inputs renders {}.

Now, also notice that on this last intent handler, I log this.$nlu...

console.log(JSON.stringify(this.$nlu, null, 4))

and there's an NlpjsNlu object attached to it, that includes a classifications array showing it was able to get the intent right, but also "entities": [], meaning it failed to extract the entity, in this case 5.

 >>>>> Request - 2021-06-22T14:24:33.114Z 
{
   "version": "3.4.0",
   "type": "jovo-platform-web",
   "request": {
      "id": "417e5bce-cde1-47b6-9bf6-41051c8f7cd0",
      "timestamp": "2021-06-22T14:24:25.032Z",
      "type": "TEXT",
      "body": {
         "text": "say 5 words of lorem ipsum"
      },
      "locale": "en",
      "data": {}
   },
   "context": {
      "device": {
         "type": "BROWSER",
         "capabilities": {
            "AUDIO": true,
            "HTML": true,
            "TEXT": true
         }
      },
      "session": {
         "id": "d9a409ec-2791-4bd2-82e3-4249d5d16010",
         "data": {}
         },
         "lastUpdatedAt": "2021-06-22T13:37:05.098Z"
      },
      "user": {
         "id": "bcb22484-1a34-4e24-94cf-d764b40f8b90",
         "data": {}
      }
   }
}

DEBUG: ****** extending jovo handler WebApp
LOG: ******************************** NEW_SESSION
DEBUG: User ID: 'bcb22484-1a34-4e24-94cf-d764b40f8b90'
LOG: ******************************** ON_REQUEST

DEBUG: 
ON_REQUEST WebApp access_token: undefined

LOG: ******************************** onResponse

 <<<<< Response - 2021-06-22T14:24:45.315Z 
{
   "version": "3.4.0",
   "actions": [
      {
         "plain": "Ok! Let's get some Lorem Ipsum. How many words do you need?",
         "ssml": "<speak>Ok! Let's get some Lorem Ipsum. How many words do you need?</speak>",
         "type": "SPEECH"
      }
   ],
   "reprompts": [
      {
         "plain": "Ok! Let's get some Lorem Ipsum. How many words do you need?",
         "ssml": "<speak>Ok! Let's get some Lorem Ipsum. How many words do you need?</speak>",
         "type": "SPEECH"
      }
   ],
   "user": {
      "data": {}
   },
   "session": {
      "data": {},
         "_JOVO_STATE_": "LoremIpsumSlots.AwaitSlots"
      },
      "end": false
   },
   "context": {
      "request": {
         "nlu": {
            "intent": {
               "name": "LoremIpsumStart"
            }
         }
      }
   }
}

 >>>>> Request - 2021-06-22T14:25:11.197Z 
{
   "version": "3.4.0",
   "type": "jovo-platform-web",
   "request": {
      "id": "85c9e579-fbe6-4dc3-9649-7d98364179d8",
      "timestamp": "2021-06-22T14:25:11.192Z",
      "type": "TEXT",
      "body": {
         "text": "5 words"
      },
      "locale": "en",
      "data": {}
   },
   "context": {
      "device": {
         "type": "BROWSER",
         "capabilities": {
            "AUDIO": true,
            "HTML": true,
            "TEXT": true
         }
      },
      "session": {
         "id": "d9a409ec-2791-4bd2-82e3-4249d5d16010",
         "data": {},
            "_JOVO_STATE_": "LoremIpsumSlots.AwaitSlots"
         },
         "lastUpdatedAt": "2021-06-22T13:37:05.098Z"
      },
      "user": {
         "id": "bcb22484-1a34-4e24-94cf-d764b40f8b90",
         "data": {}
      }
   }
}

DEBUG: ****** extending jovo handler WebApp
LOG: ******************************** NEW_SESSION
DEBUG: User ID: 'bcb22484-1a34-4e24-94cf-d764b40f8b90'
LOG: ******************************** ON_REQUEST

DEBUG: 
ON_REQUEST WebApp access_token: undefined

WARN: Inputs: {}
DEBUG: LoremIpsumResolve Inputs: {}
DEBUG: typeof this.$nlu object
DEBUG: this.$nlu.prototype undefined
DEBUG: this.$nlu.constructor function Object() { [native code] }
DEBUG: {
    "intent": {
        "name": "LoremIpsumResolve"
    },
    "NlpjsNlu": {
        "locale": "en",
        "utterance": "5 words",
        "languageGuessed": false,
        "localeIso2": "en",
        "language": "English",
        "nluAnswer": {
            "classifications": [
                {
                    "intent": "LoremIpsumResolve",
                    "score": 1
                },
                {
                    "intent": "YesIntent",
                    "score": 0
                },
                {
                    "intent": "GetForexRateStart",
                    "score": 0
                },
                {
                    "intent": "TransferStart",
                    "score": 0
                },
                {
                    "intent": "NoIntent",
                    "score": 0
                },
                {
                    "intent": "LogoutIntent",
                    "score": 0
                },
                {
                    "intent": "test",
                    "score": 0
                }
            ]
        },
        "classifications": [
            {
                "intent": "LoremIpsumResolve",
                "score": 1
            },
            {
                "intent": "YesIntent",
                "score": 0
            },
            {
                "intent": "GetForexRateStart",
                "score": 0
            },
            {
                "intent": "TransferStart",
                "score": 0
            },
            {
                "intent": "NoIntent",
                "score": 0
            },
            {
                "intent": "LogoutIntent",
                "score": 0
            },
            {
                "intent": "test",
                "score": 0
            }
        ],
        "intent": "LoremIpsumResolve",
        "score": 1,
        "domain": "default",
        "sourceEntities": [],
        "entities": [],
        "answers": [],
        "actions": [],
        "sentiment": {
            "score": 0,
            "numWords": 0,
            "numHits": 0,
            "average": 0,
            "locale": "en",
            "vote": "neutral"
        }
    }
}
LOG: ******************************** onResponse

 <<<<< Response - 2021-06-22T14:25:34.194Z 
{
   "version": "3.4.0",
   "actions": [
      {
         "plain": "How many words do you need?",
         "ssml": "<speak>How many words do you need?</speak>",
         "type": "SPEECH"
      }
   ],
   "reprompts": [
      {
         "plain": "How many words do you need?",
         "ssml": "<speak>How many words do you need?</speak>",
         "type": "SPEECH"
      }
   ],
   "user": {
      "data": {}
   },
   "session": {
      "data": {},
         "_JOVO_STATE_": "LoremIpsumSlots.AwaitSlots"
      },
      "end": false
   },
   "context": {
      "request": {
         "nlu": {
            "intent": {
               "name": "LoremIpsumResolve"
            }
         }
      }
   }
}

Hello there,

we just checked and it seems that the built-in entity extraction of nlp.js is not enabled by default.

You will need to do the following:

npm i @nlpjs/builtin-default @nlpjs/lang-en - Installs the packages that are required for the entity extraction
Make changes in the app.ts/js-file:

const { NlpjsNlu } = require('jovo-nlu-nlpjs');
const { BuiltinDefault } = require('@nlpjs/builtin-default');
const { LangEn } = require('@nlpjs/lang-en');

//
// Other content of app.ts/js
//

const nlpjsNlu = new NlpjsNlu({
  setupModelCallback: async (handleRequest, nlp) => {
    // force entity extraction
    nlp.forceNER = true;
    // register module for entity extraction
    nlp.container.use(BuiltinDefault, 'extract-builtin');
    // register module for English
    nlp.use(LangEn);
    // add corpus via path to the models directory
    // you might change it to pass an absolute path
    nlpjsNlu.addCorpus('./models');
  },
});

webApp.use(nlpjsNlu)

We will probably make this easier in the future, but for now, this workaround should work.

If you have any more questions, just let us know.

Thank you @m-ripper. I've tried your suggestion but I'm getting this error:

src/app.ts:26:12 - error TS2341: Property 'addCorpus' is private and only accessible within class 'NlpjsNlu'.
26      nlpjsNlu.addCorpus('./models');

How did you get around this?

Also, how do you declare the type of NLP.js in the model? Is this correct?

"inputs": [{
    "name": "wordCount",
    "type": { "nlpjs": "number" }
}]

Thanks,

@m-ripper, I got around the typescript compilation problem by switching to Javascript, but I'm still facing a few issues. Could you update one of the templates with a working example?

Here's what's happening:

All my requests are routed to the Unhandled handler. This only happens when the request comes from the web client. Requests from Alexa and Google Assistant work fine.
I tried creating an intent named SumsIntent that just does sums, and expects you to say something like "Add four plus five" or "Add 4 plus 5", and answers "The answer is 9" or "4 plus 5 is 9". What's happening is that only the last of the two numbers is found in the $inputs object.

Here's my adapted HelloWorld project to test this.

'use strict';

const { App } = require('jovo-framework');
const { Alexa } = require('jovo-platform-alexa');
const { GoogleAssistant } = require('jovo-platform-googleassistant');
const { JovoDebugger } = require('jovo-plugin-debugger');
const { FileDb } = require('jovo-db-filedb');
const { WebPlatform } = require ('jovo-platform-web');
const { NlpjsNlu } = require ('jovo-nlu-nlpjs');
const { BuiltinDefault } = require ('@nlpjs/builtin-default')
const { LangEn } = require ('@nlpjs/lang-en')
const app = new App()
const webPlatform = new WebPlatform();
const nlpjsNlu = new NlpjsNlu({
    setupModelCallback: async (handleRequest, nlp) => {
        // force entity extraction
        nlp.forceNER = true;
        // register module for entity extraction
        nlp.container.use(BuiltinDefault, 'extract-builtin');
        // register module for English
        nlp.use(LangEn);
        // add corpus via path to the models directory
        // you might change it to pass an absolute path
        nlpjsNlu.addCorpus('/Users/miguelangel/ws/node/my-embeddedchat/hello/models');
    },
})
webPlatform.use(nlpjsNlu);

app.use(
    new Alexa(),
    new GoogleAssistant(),
    webPlatform,
    new JovoDebugger(),
    new FileDb()
);

app.setHandler({
    LAUNCH() {
        return this.toIntent('HelloWorldIntent');
    },
    ON_REQUEST(){
        console.log(`**** ON_REQUEST Inputs: ${JSON.stringify(this.$inputs)}`) 
    },
    SumsIntent(){
        console.log(`**** SumsIntent with ${this.$inputs}`)
        let sum = parseFloat(this.$inputs.firstNumber.key) + parseFloat(this.$inputs.secondNumber.key)
        this.tell(`The answer is ${sum}`)
    },
    HelloWorldIntent() {
        this.ask("Hello World! What's your name?", 'Please tell me your name.');
    },
    MyNameIsIntent() {
        this.tell('Hey ' + this.$inputs.name.value + ', nice to meet you!');
    },
    Unhandled(){
        console.log(`**** Unhandled`)
        console.log(`this.getMappedIntentName: ${this.getMappedIntentName()}`)
        console.log(`this.getRoute: ${JSON.stringify(this.getRoute())}`)
        this.tell(`Darn! can't do that yet.`)
    }
});
module.exports = { app };

Here's my model for my SumsIntent:

{
    "name": "SumsIntent",
    "phrases": [
        "how much is ${firstNumber} plus {secondNumber}",
        "add ${firstNumber} plus {secondNumber}",
        "add ${firstNumber} and {secondNumber}",
        "${firstNumber} plus {secondNumber}"
    ],
    "inputs": [
        {
            "name": "firstNumber",
            "type": {
                "alexa": "AMAZON.NUMBER",
                "googleAssistant": "actions.type.Number"
            }
        },
        {
            "name": "secondNumber",
            "type": {
                "alexa": "AMAZON.NUMBER",
                "googleAssistant": "actions.type.Number"
            }
        }
    ]
}

Notice how I removed the "nlpjs": "number" property from the type property for both firstNumber and secondNumber. It appears to make no difference. Should I add it back? This is not documented anywhere that I could find.

Here are the request and response.

>>>>> Request - 2021-07-02T13:08:20.462Z 
{
   "version": "3.4.0",
   "type": "jovo-platform-web",
   "request": {
      "id": "c69ed9a2-8823-4376-940c-1da4efd9a858",
      "timestamp": "2021-07-02T13:08:20.385Z",
      "type": "TEXT",
      "body": {
         "text": "add 4 and 5"
      },
      "locale": "en",
      "data": {}
   },
   "context": {
      "device": {
         "type": "BROWSER",
         "capabilities": {
            "AUDIO": true,
            "HTML": true,
            "TEXT": true
         }
      },
      "session": {
         "id": "a26b810e-0b50-4528-8d8d-09a8207879da",
         "data": {},
         "new": true,
         "lastUpdatedAt": "2021-07-02T12:59:31.452Z"
      },
      "user": {
         "id": "0db85816-eab4-408d-a060-7c8098413b12",
         "data": {}
      }
   }
}

{
  start: 4,
  end: 4,
  len: 1,
  accuracy: 0.95,
  sourceText: '4',
  utteranceText: '4',
  entity: 'number',
  resolution: { value: 4, type: 'integer' }
}
{
  start: 10,
  end: 10,
  len: 1,
  accuracy: 0.95,
  sourceText: '5',
  utteranceText: '5',
  entity: 'number',
  resolution: { value: 5, type: 'integer' }
}
{
  start: 4,
  end: 4,
  len: 1,
  accuracy: 0.95,
  sourceText: '4',
  utteranceText: '4',
  entity: 'number',
  resolution: { value: 4, type: 'integer' }
}
{
  start: 10,
  end: 10,
  len: 1,
  accuracy: 0.95,
  sourceText: '5',
  utteranceText: '5',
  entity: 'number',
  resolution: { value: 5, type: 'integer' }
}
**** ON_REQUEST Inputs: {"number":{"value":"5"}}
**** Unhandled
this.getMappedIntentName: None
this.getRoute: {"intent":"None","path":"Unhandled","type":"INTENT"}

 <<<<< Response - 2021-07-02T13:08:20.517Z 
{
   "version": "3.4.0",
   "actions": [
      {
         "plain": "Darn! can't do that yet.",
         "ssml": "<speak>Darn! can't do that yet.</speak>",
         "type": "SPEECH"
      }
   ],
   "reprompts": [],
   "user": {
      "data": {}
   },

Notice how just before the **** ON_REQUEST line, Something in NLP.js or in Jovo's port for it proactively prints the entities extracted, twice. Which clearly shows that NLP.js does extract the inputs. However, on the **** ON_REQUEST you can see how only the last of the two inputs is included, and it does not bear the secondNumber key.

jovotech / jovo-starter-web-standalone

Can't extract numbers from intents. #5