Can NER value be used to drive the answer?

[MeetMartin]

corpus-en.json defines entities

"entities": {
    "topic": {
      "options": {
        "people management": ["people management"],
        "product strategy": ["product strategy"],
        "agile": ["agile", "scrum", "kanban", "agile methodology"]
      }
    }
  }

and has intent:

{
      "intent": "topic.aboutopic",
      "utterances": [
        "what is @topic"
      ],
      "answers": [
        [
          { "answer": "Agile is awesome!", "opts": "entities.topic==='agile'"},
          { "answer": "{{ topic }} is awesome!", "opts": "entities.topic==='product strategy'"}
        ]
      ]
    },

My expectation is that if user asks What is agile? the NLP gives answer Agile is awesome! based on "opts": "entities.topic==='agile'" or "opts": "topic==='agile'". But that doesn't seem to work, instead NLP returns randomly either of the answers even though the result shows entity correctly extracted as entities: [{start: 8, end: 12, len: 5, levenshtein: 0, accuracy: 1, entity: "topic", type: "enum",…}]. In addition to that {{ topic }} is not replaced in the answer by topic value and instead it is returned as raw text "{{ topic }} is awesome.

Is it not possible for NLP to choose different answers based on the NER value? Thank you for your help

[jesus-seijas-sp]

Hello, Two errors: first one, in answers, you have double square brackets, so the NLG does not reads correctly the answers. Second one is that entitites.topic is an object, with an structure like:

{
accuracy:1
end:12
entity:'topic'
len:5
levenshtein:0
option:'agile'
sourceText:'agile'
start:8
type:'enum'
utteranceText:'agile'
}

So te correct condition is entities.topic.option.

Fixed intent:

    {
      "intent": "topic.aboutopic",
      "utterances": [
        "what is @topic"
      ],
      "answers": [
        {
          "answer": "Agile is awesome!",
          "opts": "entities.topic.option === 'agile'"
        },
        {
          "answer": "{{ topic }} is awesome!",
          "opts": "entities.topic.option === 'product strategy'"
        }
      ]
    }

[MeetMartin]

thank you for your help @jesus-seijas-sp. I tried to apply the changes as you suggested but the system still seems to behave the same for some reason. Results are picked randomly without the condition applied and {{ topic }} is not replaced by the @topic value.

Do you have any other idea what might be the issue please? Otherwise I will try to create a repository for easier reproduction

[jesus-seijas-sp]

This is the test program I wrote:

conf.json

{
  "settings": {
    "nlp": {
      "corpora": ["./corpus.json"]
    }
  },
  "use": ["Basic", "LangEn"]
}

corpus.json

{
  "name": "Corpus",
  "locale": "en-US",
  "data": [
   {
      "intent": "topic.aboutopic",
      "utterances": [
        "what is @topic"
      ],
      "answers": [
        { "answer": "A: Agile is awesome!", "opts": "entities.topic.option === 'agile'" },
        { "answer": "B: {{ topic }} is awesome!", "opts": "entities.topic.option === 'product strategy'" }
      ]
    }
  ],
  "entities": {
    "topic": {
      "options": {
        "people management": ["people management"],
        "product strategy": ["product strategy"],
        "agile": ["agile", "scrum", "kanban", "agile methodology"]
      }
    }
  }
}

index.js

const { dockStart } = require('@nlpjs/basic');

const fail = () => {
  console.error('Not working');
  process.exit(1);
}

(async () => {
  const dock = await dockStart();
  const nlp = dock.get('nlp');
  await nlp.train();
  for (let i = 0; i < 100; i += 1) {
    const options = ['Agile', 'Scrum', 'Kanban'];
    const result = await nlp.process(`what is ${options[Math.floor(Math.random() * options.length)]}?`);
    if (result.answer !== 'A: Agile is awesome!') {
      fail();
    }
  }
  for (let i = 0; i < 100; i += 1) {
    const result = await nlp.process('what is Product Strategy?');
    if (result.answer !== 'B: product strategy is awesome!') {
      fail();
    }
  }
  console.log('All are correct');
})();

Is showing in console All are correct so is chosing the correct answer for each one.

Reasons why it can fail in your side: if you are not using @nlpjs/basic that register the basic plugins for you, then perhaps you don't have an Evaluator plugin. The Evaluator is the plugin responsible of evaluate the conditions of the NLG, you can see the code here: https://github.com/axa-group/nlp.js/blob/master/packages/nlg/src/nlg-manager.js#L68

If there is no Evaluator, then the NLG cannot filter the answers based on the opts, because it does not have a tool to convert those strings with javascript code to a true/false result.

[MeetMartin]

The absence of Evaluator should be it thank you. I train the model by using @nlpjs/basic but the requests are processed through import {containerBootstrap} from '@nlpjs/core'; import {Nlp} from '@nlpjs/nlp'; import {LangEn} from '@nlpjs/lang-en'; import {fs} from '@nlpjs/request';

which is then webpacked.

What is the correct wait to add Evaluator? container.use(Evaluator) doesn't seem to have an effect and container.register('Evaluator', Evaluator) leads to an error TypeError: Cannot read property 'option' of undefined

[jesus-seijas-sp]

Well, then I supose that you want it in the frontend. So you want it small... the problem with being small is that you will not want escodegen and esprima libraries in your frontend, but then you'll need to be a little "hacky".

function evalInScope(js, contextAsScope) {
  with(contextAsScope) {
    return eval(js);
  }
}

class Evaluator {
  evaluate(str, context) {
    return evalInScope(str, context);
  }
}

This class will do almost the same as the Evaluator with escodegen and esprima. with will not work in strict mode. In fact, I've put it in a function and not in the evaluate method, because classes in node are in strict mode.

Now you'll need to register this class in your container:

  container.register('Evaluator', new Evaluator(), true);

I guess that you also want the templating, so if you put *{{ entity 2 }}** is replaced by the value in the context multiplied by 2. Then you will need also this:

const evaluator = new Evaluator();

const dictionary = {};

function processString(str, context) {
  if (dictionary[str] === undefined) {
    dictionary[str] = str.match(/{{\s*([^}]+)\s*}}/g) || [];
  }
  const matches = dictionary[str];
  return matches.reduce((p, c) => {
    const solution = evaluator.evaluate(c.substr(2, c.length - 4), context);
    return solution !== null && solution !== undefined
      ? p.replace(c, solution)
      : p;
  }, str);
}

function process(obj, context) {
  if (typeof obj === 'string') {
    return processString(obj, context);
  }
  if (Array.isArray(obj)) {
    return obj.map((x) => process(x, context));
  }
  if (obj !== null && typeof obj === 'object') {
    const keys = Object.keys(obj);
    const result = {};
    for (let i = 0; i < keys.length; i += 1) {
      result[keys[i]] = process(obj[keys[i]], context);
    }
    return result;
  }
  return obj;
}

function compile(str) {
  return (context = {}) => process(str, context);
}

class Template {
  compile(str, context) {
    return compile(str)(context);
  }
}

And register this class in the IoC as a singleton:

  container.register('Template', new Template(), true);

[MeetMartin]

I will try the suggested approach, thank you

Actually, I am not trying to use it frontend so the size of the package is not a concern but I need it webpacked for Netlify Function https://docs.netlify.com/functions/build-with-javascript/ which basically deploys JavaScript into AWS Lambda. The building is done through Netlify-Lambda which uses webpack: https://github.com/netlify/netlify-lambda

I believe that the reason for webpacking in this scenario is that for a serverless function it would be inefficient to every time spring up node environment. But I'll try to research that angle some more as well.

I can build @nodejs/basic but with several warnings like

WARNING in ../node_modules/@nlpjs/core-loader/src/container-bootstrap.js 94:21-38
Critical dependency: the request of a dependency is an expression
 @ ../node_modules/@nlpjs/core-loader/src/index.js
 @ ../node_modules/@nlpjs/basic/src/index.js
 @ ./nlp.js

When the code is run it seems that dock.get('nlp') doesn't return anything.

[MeetMartin]

This seems to add the evaluator:

import {Evaluator} from '@nlpjs/evaluator';
..code..
container.register('Evaluator', new Evaluator(), true)

however processing "what is agile" leads to thrown exception ERROR: TypeError: Cannot read property 'option' of undefined

the option here refers to

{
  "answer": "Agile is awesome!",
  "opts": "entities.topic.option === 'agile'"
},

if the option is remove to be just entities.topic the response returns empty array for answers and empty for entities which means that it doesn't recognize the @topic and some other piece has to be missing as well.

[MeetMartin]

I got it working!

import {Evaluator, Template} from '@nlpjs/evaluator';
container.register('Evaluator', Evaluator);
container.register('Template', Template);
const nlp = container.get('nlp');
nlp.forceNER = true;

the forceNER was the last essential piece. Without it, the entities were not recognized so it would not also be able to use them in the evaluation.

All seems to be in order now and webpack works successfully, thank you for the help.

Down the road, I ll try to publish an article on how to get @nlpjs running for free on netlify functions. I have already published one @nlpjs article on: https://medium.com/better-programming/natural-language-processing-in-the-browser-8ca5fdf2488b

[jesus-seijas-sp]

Well, if the target is to deploy on Netifly functions, there is a blog entry and a github project: https://www.voorhoede.nl/en/blog/building-our-own-chatbot-and-cloud-ai-service/ https://github.com/Jamerrone/nlp-chatty-coffee

I see that in this project they used the full node-nlp and no webpack, but they didn't use netlify-lambda.