aichaos / rivescript-js

A RiveScript interpreter for JavaScript. RiveScript is a scripting language for chatterbots.
https://www.rivescript.com/
MIT License
377 stars 145 forks source link

We need two things, to make it perfect. #266

Open Kuchiriel opened 6 years ago

Kuchiriel commented 6 years ago

A release made up with rollup or webpack, and a .rive loader for webpack.

Almost solved the loader issue with CopyWebpackPlugin

  plugins: [
  new CopyWebpackPlugin([{
      from: path.resolve(__dirname, '../client/views/ai/brain'),
      to: path.resolve(__dirname, './dist/brain'),
      test: /\.rive$/
    }], {
      debug: 'info'
    }),
  ],

But I am getting that error

brain.js?add9:278 Uncaught (in promise) TypeError: Cannot read property 'length' of undefined at Brain._getReply (brain.js?add9:278) at Brain.reply (brain.js?add9:36) at RiveScript.replyAsync (rivescript.js?16fe:600) at VueComponent._callee3$ (index.js?1bf1:61) at tryCatch (runtime.js?4a57:65) at Generator.invoke [as _invoke] (runtime.js?4a57:303) at Generator.prototype.(:8080/anonymous function) [as next] (webpack-internal:///433:117:21) at step (index.js?1bf1:2) at eval (index.js?1bf1:2) at new Promise ()

Its something related to this line in brain.js if (this.master._sorted.thats[top].length) {

That

// constructor polyfill
if (!USE_NATIVE) {
  // 25.4.3.1 Promise(executor)
  $Promise = function Promise(executor) {
    anInstance(this, $Promise, PROMISE, '_h');
    aFunction(executor);
    Internal.call(this);
    try {
      executor(ctx($resolve, this, 1), ctx($reject, this, 1));
    } catch (err) {
      $reject.call(this, err);
    }
  };

and that

if (this.master._topics.__begin__) {
      begin = this._getReply(user, "request", "begin", 0, scope);
      if (begin.indexOf("{ok}") > -1) {
        reply = this._getReply(user, msg, "normal", 0, scope);
        begin = begin.replace(/\{ok\}/g, reply);
      }
      reply = begin;
      reply = this.processTags(user, msg, reply, [], [], 0, scope);
    } else {
      reply = this._getReply(user, msg, "normal", 0, scope);
    }

My guess, is babel-loader fucking up things (edit isn't I've tested, its probably babel-polyfill) Edit: Wasn't babel, I've tested it, and I can't remove babel-polyfill because of async await support.

kirsle commented 6 years ago

Did you call sortReplies() before trying replyAsync()? It's supposed to catch that and return an error message, but maybe if it slipped through somehow, that could be why it got an undefined variable when testing the length of that array.

this.master._sorted.thats[top].length

RiveScript._sorted is where it keeps the sorted reply cache (generated by sortReplies()), .thats is where it caches %Previous replies, indexed by topic name.

kirsle commented 6 years ago

Re: webpack.

I have the async-await branch open where I was working on converting RiveScript to use async functions and the await keyword. See PR #248 for all the benefits that will bring.

In the process I was also upgrading to CoffeeScript 2 (which outputs very nice JavaScript, which I'd eventually use to convert the code back into vanilla JS) and replacing Grunt with webpack.

I lost a bit of steam when trying to get the codebase to be backwards compatible with Node < 6. The code builds in CI for versions of Node with async/await, but I was trying to get Babel and such involved to build a backward compatible version. If you'd like to help, you can fork that branch and work on it. I haven't had a lot of free time or interest to mess with JavaScript recently.

Kuchiriel commented 6 years ago

I sorted the replies before, tried with replyAysnc() and reply(), I actually made it 'work' by removing the .length property directly from node_modules folder, but I am not really sure if it actually worked since I got my user is in a empty random topic always. I tried to force the user to be in a topic hardcoding but got no sucesss, I guess rivescript isn't seeing the .rive files properly because of some webpack transpiling 'magic' :/

I am playing with rivescript.js, trying to port it to ECMA 8, got a lot of progresss, but I think the problem will be testing everything when I finallly finish, if I succeed I will pulll request with tests, if you want to, just create a rollup/webpack ecma8 branch for rivescript.js when I finish.

kirsle commented 6 years ago

To help debug it, try doing console.log(JSON.stringify(rs._topics, null, 2)) to print out the _topics structure.

I'm starting to think it was failing to load the replies in the first place! Also if you turn on debug mode and don't see it parsing all your rivescript files, that could be it.

I actually got inspired to work more on my async/await branch today, so progress on that will be coming along. I'm getting rid of CoffeeScript in the process! Going full ES2015+ :)

Kuchiriel commented 6 years ago

rs.topics[..] outputs {} Debug Mode looks fine image

kirsle commented 6 years ago

Going back to your original ticket:

this.master._sorted.thats[top].length

Are you making sure to call sortReplies() between when you load the .rive files and before you get a reply?

The _sorted.thats structure should get populated for every topic defined in your RiveScript code with the default "random" topic always being there.

Here's an example for how to load RiveScript properly:

let bot = new RiveScript();

// loadDirectory and loadFile are async so resolve their Promise
// and that's when you know they've finished loading.
bot.loadDirectory("./brain").then( () => {
    bot.sortReplies(); // only after the replies were loaded,
                       // here, in the promise.then function

    // now you're ok to get a reply
    bot.reply("user", "hello bot").then( (reply) => {
        console.log("Bot>", reply);
    });
});

If you were to comment out the bot.sortReplies() this script would probably reproduce the same error in your original ticket when run as a simple vanilla Node.js script.

Try console.log(bot._sorted, null, 2) to dump the entire sorted topic tree structure. There should be objects in there like "topics" and "thats" with keys named after topics and arrays of sorted triggers and such. If those aren't there, you didn't call sortReplies().

Kuchiriel commented 6 years ago

Since you locked the PR I will answer here.

My usage of Webpack in this repo shouldn't have any effect on what other developers are doing with my module. I use Webpack to produce the dist/rivescript.min.js for easy access for embedding in a web browser. If a third-party JavaScript developer has listed rivescript in their package.json dependencies and they have their own Webpack stack for building their app's bundle.min.js or whatever -- my Webpack config doesn't have anything to do with theirs.

Look my PR and see what I did with webpack.config.js and makefile, you will understand what I did.

kirsle commented 6 years ago

Hi,

To help me verify what exactly the problem is, can you give me a working test case? Maybe as a GitHub gist or link me to a branch on your repo.

I think if you were to copy my example code above into a new .js file (and your .rive files are in ./brain) it would work without giving you that error. From there, make it more complex and closer to your current stack until you start to reproduce the problem and then let me see what you did.

Kuchiriel commented 6 years ago

Hello

Ok I will upload to my github and edit this comment. I will upload a example project, not my original one (because of hardcoded things, api tokens, password and etc), but the behavior occurs the same in both.

@kirsle https://github.com/Kuchiriel/example

Any help to run the project just ask.

Kuchiriel commented 6 years ago

See here https://github.com/Kuchiriel/example/blob/master/client/views/ai/index.js

I will try to run again, since you made a lot of modifications, this issue is old.

Oh sorry, i didn't mention, you with interact with the bot by speech (Chrome/Chromium), I did not put other interface, you may need to put one or hardcode to get replies. (Now there is an interface)

I am editing to put an interface and make sure I am sorting replies.

kirsle commented 6 years ago
    await this.bot.loadFile([
        './brain/begin.rive',
        './brain/sarcasm.rive',
        './brain/about.rive',
        // web,
        // backend,
        // about,
        // search,
        // spotify
      ],
      this.bot.sortReplies() && console.log('RiveScript: Replies Sorted'),
      on_load_error)

I think that the line this.bot.sortReplies() && console.log('RiveScript: Replies Sorted') is not correct.

You're using loadFile in the old callback-style way (loadFile(files, onSuccess, onError)), and instead of a function object given to the onSuccess parameter, you are calling sortReplies() && console.log which I imagine will evaluate to true and not even be a function call. The await keyword, however, might wait for loadFile() to finish and continue running your code -- but sortReplies() had been called at the wrong moment.

Try this instead:

    // `await` will already wait until the Promise resolves
    // before continuing with your code. If the promise throws
    // a rejection, this can be caught with try/catch when using
    // async/await.
    await this.bot.loadFile([
        './brain/begin.rive',
        './brain/sarcasm.rive',
        './brain/about.rive',
        // web,
        // backend,
        // about,
        // search,
        // spotify
      ]);

      // here, loadFiles() has blocked until it resolved and you
      // can sort the replies now.
      this.bot.sortReplies() && console.log('RiveScript: Replies Sorted');

Another example of how async/await works: when you await on a function that returns a Promise, the execution of the code "pauses" there until the Promise resolves and the resolved value is returned like a normal value from the function. If the Promise rejects instead, it raises an Exception that you catch using try/catch.

// The reply() function returns a Promise, so when you `await` it you can
// get the string reply and catch possible exceptions in a way that resembles
// normal sync code.
var reply;
try {
    reply = bot.reply(username, message);
} catch(e) {
    reply = "I ran into an error: " + e;
}
console.log("Bot>", reply);

/////////////////////////

// (when not using async functions or await, the Promise version
// of the above looked like this):
bot.reply(username, message).then(function(reply) {
    console.log("Bot>", reply);
}).catch(function(error) {
    console.log("Bot>", error);
});
Kuchiriel commented 6 years ago

Right, I will try and give a feedback.

Now I got this.

{} index.js:35:10
Asked to reply to [Hi] undefined rivescript.js:113
You forgot to call sortReplies()!

I will put a try catch and see the result

Same error and this

Unhandled promise rejection TypeError: "this.master._users[user] is undefined" onAfterReplywebpack-internal:///215:60:5replywebpack-internal:///215:48:7replywebpack-internal:///213:595:12_callee4$webpack-internal:///119:189:24tryCatchwebpack-internal:///433:65:37invokewebpack-internal:///433:303:22methodwebpack-internal:///433:117:16stepwebpack-internal:///119:17:183_asyncToGeneratorwebpack-internal:///119:17:437Promisewebpack-internal:///347:177:7_asyncToGeneratorwebpack-internal:///119:17:99recogFunctionwebpack-internal:///119:205:16_callee$webpack-internal:///119:61:22tryCatchwebpack-internal:///433:65:37invokewebpack-internal:///433:303:22methodwebpack-internal:///433:117:16stepwebpack-internal:///119:17:183stepwebpack-internal:///119:17:361runwebpack-internal:///347:75:22notifywebpack-internal:///347:92:30flushwebpack-internal:///102:18:9

Kuchiriel commented 6 years ago

Man I will push my modifications, I manage to sort the replies now, BUT I KEEP IN A EMPTY TOPIC NAMED RANDOM, that is the issue, and the debugger says the .rives are properly loaded.

RiveScript Interpreter v1.19.0 Initialized. 213:113:14 <- I guess I need to upgrade this. Runtime Environment: web 213:113:14 Sorting triggers... 213:113:14 RiveScript: Replies Sorted 119:56:41 {} 119:58:22

ERR: No default topic 'random' was found!

@kirsle Pushed.

I will upgrade every dep try to run and push again.

kirsle commented 6 years ago

ERR: No default topic 'random' was found!

This usually means the replies didn't load properly. I saw in your console output you had an empty object {}, this looks like when you tried to console.log(JSON.stringify(bot._topics).

Ideas on what to do next:

1) Turn on debug mode and examine all the output.

Edit: by providing debug: true to the constructor options, like new RiveScript({ debug: true });

The debug log should show you two things:

e.g.:

Cmd: +; line: everyone *
    Trigger pattern: everyone *

e.g.

Sorting triggers...
Analyzing topic random...
Collecting trigger list for topic random (depth=0; inheritance=0; inherited=0)
Collecting trigger list for topic random (depth=0; inheritance=0; inherited=0)

2) Log the _topics and _sorted structure.

console.log(JSON.stringify(bot._topics, null, 2));
console.log(JSON.stringify(bot._sorted, null, 2));

Example bot._topics structure should show lists of triggers grouped under their topics, like

/eval console.log(JSON.stringify(bot._topics, null, 2));
{
  "random": [
    {
      "trigger": "shutdown{weight=10000}",
      "reply": [
        "{@botmaster only}"
      ],
      "condition": [
        "<id> eq <bot master> => Shutting down... <call>shutdown</call>"
      ],
      "redirect": null,
      "previous": null
    },
    {
      "trigger": "botmaster only",
      "reply": [
        "This command can only be used by my botmaster. <id> != <bot master>"
      ],
      "condition": [],
      "redirect": null,
      "previous": null
    },
    {
      "trigger": "coffee test",
      "reply": [
        "Testing CoffeeScript object: <call>coffeetest</call>"
      ],
      "condition": [],
      "redirect": null,
      "previous": null
    },

_sorted should similarly show a large data structure of triggers sorted under their topics:

{
  "topics": {
    "random": [
      [
        "shutdown{weight=10000}",
        {
          "trigger": "shutdown{weight=10000}",
          "reply": [
            "{@botmaster only}"
          ],
          "condition": [
            "<id> eq <bot master> => Shutting down... <call>shutdown</call>"
          ],
          "redirect": null,
          "previous": null
        }
      ],
      [
        "(what is my name|who am i|do you know my name|do you know who i am){weight=10}",
        {
          "trigger": "(what is my name|who am i|do you know my name|do you know who i am){weight=10}",
          "reply": [
            "Your name is <get name>.",
            "You told me your name is <get name>.",
            "Aren't you <get name>?"
          ],
          "condition": [],
          "redirect": null,
          "previous": null
        }
      ],

If you don't see large data structures in both places, it's a hint that your data wasn't loaded correctly. Double check the paths on disk, etc. and verify it loaded the code and that sortReplies() was called at the right time.

Of course, it is possible that you just don't have any replies in the "random" topic, but I think that's unlikely; but if the RiveScript source you loaded has like > topic something surrounding every single trigger defined so that there are zero triggers in the default topic, it would still trigger the "Empty topic 'random'" error, but the vast majority of the time the root cause of that error is that the *.rive sources weren't loaded successfully.

Kuchiriel commented 6 years ago

I upgraded the whole thing, now Webpack is messing with me about vue-loader xD I will follow your advices, fix the vue-loader trouble and feedback.

@kirsle Fixed Webpack3 to Webpack4 issues, pushed my modifications, this is the console output.

RiveScript Interpreter v2.0.0-alpha.6 Initialized. 316:325:12 Runtime Environment: web 316:325:12 Sorting triggers... 316:325:12 RiveScript: Replies Sorted 101:66:41 Bot topics: {} 101:68:22 Bot sorted: undefined index.js:37:10 Asked to reply to [Hi] undefined User Hi was in an empty topic named 'random' Bot ERR: No default topic 'random' was found!

I guess I found the problem. I will update if I manage to fix. (I was sending the message parameter without the user parameter LoL)

Anyway same ghost, Bot ERR: No default topic 'random' was found

RiveScript Interpreter v2.0.0-alpha.6 Initialized. 316:325:12 Runtime Environment: web 316:325:12 Sorting triggers... 316:325:12 RiveScript: Replies Sorted 101:66:41 Bot topics: {} 101:68:22 Bot sorted: undefined index.js:37:10 Asked to reply to [Trevor] Hi

User Trevor was in an empty topic named 'random'

Kuchiriel commented 6 years ago

@kirsle

The trouble is the Vue context. Rivescript inside Vue context don't work properly.

This by example

this.bot.loadFile([])
.then(onReady)
.catch(onError)

Edit: Even outside Vue context, the code above does not work. Edit: Manage to work by taking the functions (onReady, onError) out of a variable, it need to be explicit declared.

If we force hardcoding to sort the replies, Rivescript is unable to manage the user context and the user always will be in the topic 'random' which Rivescript does not see (because its inside the Vue context), therefore, can't get an answer for the query, since it does not see the user message either.


[Maggie]: TypeError: Cannot read property 'length' of undefined
    at Brain._getReply (brain.js?cec5:182)

This is not a Webpack compatibilty issue, this was solved with Webpack Copy Plugin, the real problem is Vue.js compatibility.

If I run Rivescript outside the Vue context, it works flawless.

So the improvement here will be make it work inside Vue context.

I don't know if is needed to change the project code, or rivescript code. I will be doing tests, if I suceed you may create a Vue.js e.g.

Pushed modifications.

Kuchiriel commented 6 years ago

Working now.

This is not a solution just a workaround for people that uses Vue.js, you need to put rivescript outside the Vue context.

The problem is, you can't restrict the bot by Vue elements, if someone have a better idea to work with rivescript and Vue.js please comment here.

e.g

import rivescript from 'rivescript'

const bot = new rivescript({
  debug: false,
  utf8: false
})

bot.loadFile([
    './brain/begin.rive',
    './brain/sarcasm.rive',
    './brain/about.rive',
    // web,
    // backend,
    // about,
    // search,
    // spotify
  ])
  .then(onReady)
  .catch(onError)

function onReady(msg) {
  bot.sortReplies() && console.log('RiveScript: Replies Sorted', msg)
  this.botReply = msg
}

function onError(err) {
  console.log(`[${this.botName}]:`, err)
  this.botReply = err
}

export default {
  data() {
    return {
      botName: 'Maggie',
      botMaster: 'Trevor',
      yourMessage: 'Hi',
      botReply: 'Hey talk to me!',
    }
  },
  mounted() {
    this.talkToBot(this.yourMessage)
  },
  methods: {
    talkToBot(message) {
      this.yourMessage = message
      const onSuccess = reply => {
        console.log(`[${this.botName}]:`, reply)
        this.botReply = reply
      }
      const onError = err => {
        console.log(`[${this.botName}]:`, err)
        this.botReply = err
      }
      this.reply = bot.reply(this.botMaster, message)
        .then(onSuccess)
        .catch(onError)
    }
  }
}
kjleitz commented 6 years ago

@Kuchiriel I use Vue, Webpack, and Rivescript together in a project of mine. If it helps, this is my simple setup: I have the engine initialize, load files, and sort replies in its own file/module, and then I export the engine instance. Wherever I need the engine, I import it. It'll be the same instance anywhere you import it, so you don't have to re-initialize/load/sort each time you want to use it.

Since loadFile() is asynchronous, remember that by the time your component here mounts, bot may not yet have loaded the files and called the callback containing sortReplies(), depending on the speed at which it does so. If you really need to send a message when the component mounts, you could always implement a kind of "queueing" pattern. I'll comment with how I do that in a bit.

kjleitz commented 6 years ago

This is a simplified version of the pattern I use:

// this can be found in 'bot/bot_engine'

import RiveScript from 'rivescript';
import config from 'bot/config';

class BotEngine {
  queue(onSuccess = () => {}, onError = () => {}) {
    if (this._engine) {
      onSuccess(this._engine);
      return;
    }
    const rsEngine = new RiveScript();
    rsEngine.loadFile(config.brainFiles).then(() => {
      rsEngine.sortReplies();
      this._engine = rsEngine;
      onSuccess(this._engine);
    }).catch((error) => {
      console.warn(`Error loading brainfiles: ${error}`);
      if (this._engine) delete this._engine;
      onError(error);
    });
  }
}

const botEngine = new BotEngine();

export default botEngine;

The BotEngine class allows me to wrap the engine so that I can pass messages to it without worrying if everything's been initialized yet. An example of its use in a Vue component might be like this:

<template>
  <div>
    <ul class="responses">
      <li v-for="message in messages">{{message}}</li>
    </ul>
    <input v-model="inputMessage" @keyup.enter="sendMessage"/>
  </div>
</template>

<script>
import botEngine from 'bot/bot_engine';

export default {
  data() {
    return {
      messages: [],
      inputMessage: '',
    };
  },

  methods: {
    sendMessage() {
      this.messages.push(this.inputMessage);
      botEngine.queue((engine) => {
        engine.reply('keegan', this.inputMessage, this).then((outputMessage) => {
          this.messages.push(outputMessage);
        });
        this.inputMessage = '';
      });
    },
  },
};
</script>

You probably don't even need the queue, really, as long as you're not immediately sending messages to the bot before it has a chance to asynchronously load the files and sort the replies... but if you are, then that might be a pattern you should consider. Hope it helps.

Kuchiriel commented 6 years ago

@kjleitz It really helped! Thanks a lot!

IdealPress commented 4 years ago

Hi @kjleitz & @Kuchiriel – I realise this is an old thread but am hoping someone can help me out with a quick question. @kjleitz, when you call import config from 'bot/config'; and rsEngine.loadFile(config.brainFiles).then(() => {... – what is it specifically that you are importing/loading? Is this a directory of .rive files or a particular config object or file? Apologies for the basic question, I'm pretty new to webpack, etc. 🙏

kjleitz commented 4 years ago

@IdealPress no worries! I left config kind of ambiguous. I was imagining that you'd make a separate file (bot/config.js) for your project configuration, and you'd have an array of URIs (which point to your .rive files) there in a property called brainFiles. Like this:

// in bot/config.js

const config = {
  // ...other config...

  brainFiles: [
    // These are your `.rive` files; if you're in the browser it'll try to get them
    // over the network from "https://www.your.website/brain/begin.rive", and then
    // "https://www.your.website/brain/main.rive", and so on (you get the picture)
    '/brain/begin.rive',
    '/brain/main.rive',
    '/brain/some_other_stuff.rive',
    '/brain/et_cetera.rive',
  ],

  // ...other config...
};

export default config;

...but you don't have to do it that way if you don't have a separate config-like object in your project. You could just as easily plant it right in place, like:

// in bot/bot_engine.js

// ...

class BotEngine {
  queue(onSuccess = () => {}, onError = () => {}) {
    // ...
    rsEngine.loadFile([
      // These are your `.rive` files; if you're in the browser it'll try to get them
      // over the network from "https://www.your.website/brain/begin.rive", and then
      // "https://www.your.website/brain/main.rive", and so on (you get the picture)
      '/brain/begin.rive',
      '/brain/main.rive',
      '/brain/some_other_stuff.rive',
      '/brain/et_cetera.rive',
    ]).then(() => {
      // ...
    }
  }
}

// ...

Hope that helps!