Patterns with wildcard in the beginning is not parsed/matched

[lordhari]

Patterns like "_ I am having issues with Program-O" are not matched with "Hi, I am having issues with Program-O".

At the same time, patterns like "I am having issues with _" are matched for "I am having issues with the chatbot". Later I found none of the patterns with wildcard in the beginning is working. They used to work before.

[Dave-Morton]

I'll have to look into this over the weekend. Thanks for bringing it to our attention. Are you seeing any errors popping up in the logs?

[lordhari]

There is RSS load failed. Error: String could not be parsed as XML in rss.error.log. Right after the pattern beginning with wildcard is added. Nothing else.

[Dave-Morton]

Hmmm... I thought we took care of the RSS error a couple of versions ago. Do you have the latest version of Program O (updated yesterday) installed?

[lordhari]

Yup, you are right. That was the log for the older version. Anyway the problem persists for both versions (for yesterday's version also).

Errors in the log of v2.5.4 are

PHP ERROR [Warning] -Wrong parameter count for strstr() in /programo/chatbot/core/aiml/find_aiml.php on Line 100 PHP ERROR [Warning] -Wrong parameter count for strstr() in /programo/chatbot/core/aiml/find_aiml.php on Line 91 PHP ERROR [Warning] -preg_match_all() expects at least 3 parameters, 2 given in /programo/chatbot/core/aiml/find_aiml.php on Line 208

The first two is popping together for 9 times then the last error is logged 100+ times.

[Dave-Morton]

Ok, that's a good starting point. I won't be able to start work on it till tomorrow, but it's high on the ToDo list.

Just to be clear, though, patterns like the following:

_ [word match]
* [word match]

don't work at all, regardless of the contents of [word match], while these:

[word match] _
[word match] *
[word match] _ [word match]
[word match] * [word match]

all seem to?

[lordhari]

I am sorry. But I had them working. This aiml file have a bunch of them. How is that so?

[Dave-Morton]

Can you provide me with a couple of examples of categories that are confirmed not to work? That will simplify matters.

[lordhari]

Like I said in the beginning, "_ I am having issues with Program-O" is not working.

[Dave-Morton]

Ok, I'll be sure to include that in my testing. Thanks.

[Dave-Morton]

Um... Looking at your pattern now that I have time, I can see one problem right off the bat. That hyphen (dash, minus sign, whatever) is considered to be punctuation in AIML parlance, and is not allowed. the proper pattern (as stored in the database) needs to be this:

_ I AM HAVING ISSUES WITH PROGRAM O

Please note that the pattern has been "normalized" into all uppercase text, with only letters, spaces and the underscore wildcard. Please correct your pattern to remove the hyphen and try again. I just spent several hours testing this and can find no problem unless I leave that hyphen in. THEN things go to heck.

[lordhari]

Ok, the thing is, everything works perfectly when the database is empty other than the admin_added.aiml. When I add all the other necessary aiml files (got from alicebot.org), issues start to pop up.

So, I started adding aiml files one by one and tested for problems. After about half of all the files I got, when I uploaded this one, I started having problems with patterns with tag.

Here's an example: `

I WOULD LIKE TO APPLY FOR A JOB

`

<topic name="vacancy"><category><pattern>*</pattern><that>What skill sets do you have that would fit with us?</that><template>Ok, I will pass on the request.</template></category></topic>

The second category is not matched with the user reply. The weird thing is, it worked well before uploading that particular aiml file and the bot would reply "Ok, I will pass on the request". Also after aiml categories in it are cleared.

The errors generated are the same as I commented above. What wrong can you see in that aiml file? Default.aiml contains around 4K very useful aiml categories.

[Dave-Morton]

Have you tried running the built-in AIML validator to make sure that the file is a valid AIML file? The upload script only makes sure that uploaded files are "well formed" XML, and doesn't do any testing to make sure that files are actually valid AIML. This is up to the botmaster (e.g. you) to do. In that file are at least 4 errors that need to be addressed before it can be used. Those errors are on lines 867, 1622, 3490 and 4000, respectively. To validate the files yourself, navigate to the AIML upload page and read the paragraph at the top of the page. You'll see a link in there to the AIML validator.

[lordhari]

Sorry for the delay. Was down with fever. I get that at line 867, deprecated get_likes is the problem. But I couldn't find what is wrong at lines 1622, 3490 and 4000. Anyway, I deleted the categories at those lines and tested with the AIML validator. But it still showed File Default.aiml is NOT valid!.

Warning 1549: failed to load external entity "aiml.dtd" on line 0, column 0

Error 517: Could not load the external subset "aiml.dtd" on line 0, column 0

But lines that have errors are not shown in the validator. How can we find that in the first place? Also many other aiml files(that doesn't conflict with my aiml categories) I uploaded before was shown not valid by the validator.

[Dave-Morton]

the problems with the other ones was simply that they had a period in the wrong place (inside the <random> tag, instead of outside of it). It's interesting that your errors all show line 0 for the errors, whereas mine did not. Here is the output from the validation script for the original default.aiml file:

Error 515: Element get_likes is not declared in template list of possible children on line 867, column 0
<category><pattern>DO I LIKE *</pattern><template>I know you like <get_likes/>.</template></category>
Error 534: No declaration for element get_likes on line 867, column 0
<category><pattern>DO I LIKE *</pattern><template>I know you like <get_likes/>.</template></category>
Error 504: Element random content does not follow the DTD, expecting (li)+, got (li li li CDATA) on line 1622, column 0
<category><pattern>HOW MANY _ DO I HAVE</pattern><template> <random> <li>How would I know how many?</li> <li>I don't know how many you have.</li> <li>I know you have five fingers.</li> </random> <random> <li>Let me guess...</li> <li>I will try...</li> <li>I can imagine...</li> </random> <random> <li>three</li> <li>five</li> <li>seven</li>. </random> <think> <set name="it"> <set name="topic"> <person/> </set> </set> </think> </template></category>
Error 504: Element random content does not follow the DTD, expecting (li)+, got (li li li li li li li CDATA) on line 3490, column 0
<category><pattern>WHAT COLOR ARE *</pattern><template> <random> <li>Red</li> <li>Green</li> <li>Yellow</li> <li>Blue</li> <li>gray</li> <li>Black</li> <li>white</li>. </random> </template></category>
Error 504: Element random content does not follow the DTD, expecting (li)+, got (li li li li li li li CDATA) on line 4000, column 0
<category><pattern>WHAT IS THE LARGEST *</pattern><template> <random><li>Asia.</li><li>A Whale.</li><li>The Grand Canyoni></li><li>Jupiter</li><li>St. Peter's in Rome</li><li>There is no largest <person/></li><li>China</li>.</random></template> </category> 

As you can see, the line numbers are shown in those error messages.

If you wish, you can download the complete ALICE AIML set with all files corrected at http://www.geekcavecreations.com/?downloads (look for aiml.zip - it's usually in the lower-left corner of the grid of files). Perhaps you'll have better luck with those files.

[lordhari]

Wow! This must be bad luck. I am unable to download the aiml.zip.

Connection failed: SQLSTATE[HY000] [2005]

Unknown MySQL server host 'gccdbgen.db.7044252.hostedresource.com' (0) Warning: Cannot modify header information - headers already sent by (output started at /home/gcc1multi/public_html/downloadMe.php:34) in /home/gcc1multi/public_html/downloadMe.php on line 42 Warning: Cannot modify header information - headers already sent by (output started at /home/gcc1multi/public_html/downloadMe.php:34) in /home/gcc1multi/public_html/downloadMe.php on line 43

[Dave-Morton]

You... broke my... site??? O.O

Kidding. I'll attach it here, then look into why my downloads page isn't working properly.

aiml.zip

[lordhari]

All of the AIML files in the zip you attached here is shown as invalid in the AIML validator.

It is either

Warning 1549: failed to load external entity "aiml.dtd" on line 0, column 0 Error 517: Could not load the external subset "aiml.dtd" on line 0, column 0

or

Fatal Error 65: Space required after the Public Identifier on line 2, column 59 Fatal Error 43: SystemLiteral " or ' expected on line 2, column 59 Fatal Error 70: SYSTEM or PUBLIC, the URI is missing on line 2, column 59

Both aren't particularly helpful. Problem with the validator? I went ahead, cleared the logs, cleared all other aiml categories (purge database) and uploaded the aiml files. No error logs generated. But after couple of queries, bot started replying only the error response message.

PHP ERROR [Warning] -Wrong parameter count for strstr() in /programo/chatbot/core/aiml/find_aiml.php on Line 100 PHP ERROR [Warning] -Wrong parameter count for strstr() in /programo/chatbot/core/aiml/find_aiml.php on Line 91 PHP ERROR [Warning] -preg_match_all() expects at least 3 parameters, 2 given in /programo/chatbot/core/aiml/find_aiml.php on Line 208

[Dave-Morton]

Interesting. I'll have to not only fix my website this weekend, but replace that zip file, as well. I'm attaching a new zip archive here. In that archive are 61 AIML files that I've personally just run through the validator and packaged. If you have trouble after this, I'll need some very specific information about your system so that I can do some direct investigation of the problem. But first, try these AIML files:

ALICE_Corrected_Valid.zip

[lordhari]

PHP Notice: Use of undefined constant DIR - assumed ' DIR ' in /programo/admin/validateAIML.php on line 13 PHP Warning: chdir(): No such file or directory (errno 2) in /programo/admin/validateAIML.php on line 13

What should be the value of DIR in validateAIML.php?

[lordhari]

Yup. Still the same. The bot breaks when the pattern with that tag is queried. Thereafter every reply is the error response message. Reloading the session will only fixes it. Errors generated are the same:

PHP ERROR [Warning] -Wrong parameter count for strstr() in /programo/chatbot/core/aiml/find_aiml.php on Line 100 PHP ERROR [Warning] -Wrong parameter count for strstr() in /programo/chatbot/core/aiml/find_aiml.php on Line 91 PHP ERROR [Warning] -preg_match_all() expects at least 3 parameters, 2 given in /programo/chatbot/core/aiml/find_aiml.php on Line 208

Bot is hosted in a bluehost server. cPanel -11.52.6 Apache -2.2.31 PHP -5.2.17 MySQL -5.5.42 Perl -5.10.1 Any other info you need?

[Dave-Morton]

__DIR__ is what is called a magic constant in PHP, so getting that error is telling me that your PHP version is too old to allow Program O to work correctly. Program O requires PHP 5.3.0 or above (the same version of PHP that introduced __DIR__) in order to work. I suggest contacting BlueHost to see if they can provide a newer version of PHP. If they cannot, then the next step is to find another hosting service. Sadly, this means that until your PHP is upgraded there's nothing more that I can do to help.

[Dave-Morton]

I just went back through the code for the install script, as well as the README file, and I noticed that the PHP version requirement was either not updated properly (install_programo.php) or simply not listed (README). I've fixed that, and I apologize for any inconvenience that this may have caused. Program O has actually required PHP version 5.3 for a while now, and the version requirements should have been updated way back then. Oops?

[lordhari]

That solved the problem with AIML validator. Also, Program O is now error free. But! The problem with the pattern still persists. No errors. AIML code I used is in a comment above. Do you find any wrong in it? Current versions: cPanel -11.52.6 Apache -2.2.31 PHP -5.4.43 MySQL -5.5.42 Perl -5.10.1

[Dave-Morton]

Ok, now we can investigate the issue properly, without any problems with older versions getting in the way. First off, how is your install of Program O configured with regard to debugging? The settings for debugging are found in the admin under the "Current Bot" panel, on the right side of that panel. The two relevant settings are "Debug Mode", which should be set to file, and "Debug Level", which should be set to "show everything". Once those two settings are configured that way, we can then do some proper testing. I'm writing an AIML file that is specifically designed to test this issue, and will post it here once it's completed (and tested). Once posted, please create a new testing chatbot and load only this AIML file into it, then test it using the user inputs I'll provide in the same post as the file. then load in the AIML set that you've downloaded from my earlier post and test again, posting the chat outputs from each run. then I'll be able to see the specific problem, and can compare the outputs with what I got on my system. This should help us to narrow down where the trouble is.

[lordhari]

Current bot have the recommended settings. Debug Mode: file log - log debugging to a file Debug Level: show everything.

[Dave-Morton]

Good. I'll probably need to see some debug files at some point. :)

While writing the AIML file for testing, I went back through all of the previous posts here, to gather any AIML categories listed in order to make use of them (or similar~ish versions), and I ran across this:

<category> <pattern>I WOULD LIKE TO APPLY FOR A JOB</pattern> <template><think><set name="topic">vacancy</set></think> What skill sets do you have that would fit with us? </template> </category>

<topic name="vacancy"><category><pattern>*</pattern><that>What skill sets do you have that would fit with us?</that><template>Ok, I will pass on the request.</template></category></topic>

This is a problem, so I thought I would clarify something about AIML in general, that affects the way Program O works, and that's this:

All pattern-side AIML elements (e.g. <pattern> & <that> must contain only letters, numbers, spaces and/or wildcards. Special characters other than _ or * are considered punctuation, and are not allowed.

In the category, above, there is a question mark in the <that> pattern, and that will cause Program O to completely ignore that entire category, just as if it weren't there. there is no way for the AIML validator to detect these problems, since the DTD (Document Type Declaration - a collection of definitions and rules that dictate what constitutes a "valid" document) that it checks against (which I did not write) is not that specific, so it's up to the botmaster to know and obey these rules on their own. I'm looking into possibly re-writing the DTD to catch these types of problems, and I'm considering adding an option to validate AIML created by the teaching interface, but these things are mid- to long-term goals, probably for version 3, which is still a long way off, so for now, a good working knowledge of AIML syntax is required of any botmaster.

[lordhari]

Yup. That solved the issue. Seems alright in my quick tests.

But I can't test all types of my patterns today itself. Will bug you if I find anything.

[Dave-Morton]

Great. Glad that helped.

On a side note, some preliminary research shows that in order to restrict the contents of a "valid" AIML pattern to words, numbers, spaces or wildcards, using a simple(lol) DTD won't work. I'll probably have to switch over to using an XML Schema (XSD), instead, which will require a different approach than what's currently in use, so that means that this won't happen any time soon. Shame, really, but I'm sure I'll get around to it at some point. :)