using old version of wit.ai api

[dfischer]

With wit.ai new api it's no longer appropriate to classify things off of the intent attribute.

It seems there has to be another strategy to figure it out.

---

Investigating this.

[dfischer]

Well, actually having a bit of trouble on how to do this the right way. It seems with the new api they expect to handle a conversational flow with runActions and it doesn't seem really possible to interact howdy with that.

---

Might still be able to do a low level api hit on the intent of a message but that doesn't capture the whole point of wit.

[dfischer]

Well, I pulled it off the original way this middleware was working. If people are expecting to use the stories functionality that's going to take a lower level api and try to figure out how to use the converse feature which ... kind of defeats the purpose of howdy to a point I think. But I'd have to think about it more.

For the low level approach to just matching off of entities/intents (you'll have to create an intent entity) then I put something together which is available here:

https://github.com/dfischer/botkit-middleware-witai/commit/f6117c6adb66781540daa69d710a2764a3bef9a6

One caveat is that on my testing, confidence always came back as null; but maybe that's because of my training data set.

[darkin1]

Yes, I spoke with Stepan from wit.ai and he told me that message in API it's deprecated

curl -XGET 'https://api.wit.ai/message?v=20140916&q=hi' \
-H 'Authorization: Bearer my_token'
{
  "msg_id" : "xxxxxx-dff3-486a-9bb3-f11b26f7f68e",
  "_text" : "hi",
  "outcomes" : [ {
    "_text" : "hi",
    "confidence" : null,
    "intent" : "default_intent",
    "entities" : {
      "intent" : [ {
        "value" : "welcome"
      } ]
    }
  } ],
  "WARNING" : "DEPRECATED"

Now, the right way it's to use converse endpoint https://wit.ai/docs/http/20160330#converse-link

[darkin1]

@dfischer I noticed that in your code actions it's missing

ReferenceError: actions is not defined

[dfischer]

Woops, good catch sorry @darkin1. (I was just using boilerplate object from wit.ai docs for reference)

Yeah, interesting about the converse endpoint. I'm not sure what the flow is supposed to be with howdy in order for that to play appropriately. You'd have to loop within a trigger? I tried to integrate howdy flow with converse but I couldn't make sense of it, so I went back to the old way of mimicking intents through entities.

[darkin1]

I created testing bot using your source code for wit.ai, it works pretty well. I didn't have time to test conversation approach, but when I do I will inform you.

[dfischer]

Updated examples: https://github.com/dfischer/botkit-middleware-witai

Works for me.

I'd like to see the converse interface with howdy, but still, not sure the best architecture for that.

[dfischer]

Well I think they just updated their api and broke some stuff – seems that entities is no longer in outcomes? That kind of rubbed me the wrong way, good thing stuff isn't on prod right now.

@darkin1 do you see a way to use the converse api that meshes well with howdy?

[darkin1]

@dfischer I see that they change response. Some quick fix and it's work

        if (message.text ) {
            client.message(message.text, function(error, data) {

                if (error) {
                    next(error);
                } else {
                    // not sure how to handle multiple outcomes right now
                    message.entities = data.entities;
                    next();
                }
            });

And i noticed that confidence works fine now

contact: 
bot_1    | 17:54:18 bot-0    [ { body: '<@U056LAGRM>',
bot_1    | 17:54:18 bot-0        value: [Object],
bot_1    | 17:54:18 bot-0        confidence: 0.9678939346906366,
bot_1    | 17:54:18 bot-0        start: 6,
bot_1    | 17:54:18 bot-0        end: 18,
bot_1    | 17:54:18 bot-0        entity: 'contact' } ] }

[darkin1]

@dfischer

I did some tests and convers API it's very confusing. I know how to get intent from query (question). You can use it like message endpoint and it's work pretty well.

I think that intention of using convers enpoint it's only to move some logic from botkit to web.ai (using stories). But looking at the previous changes I suppose that web.ai API will often vary and there is no need to use advanced options.

However if you want get some action after another action it could be usefull, like this example

u: What's is the weather
b: in Where
u: Warsaw
b: i need check forecast API
b: {execute some action from API}
b: it's sunny

When I checked basic examples from wit.ai I get different results and I don't exactly understand why. I set story exacly as in the second example (https://wit.ai/docs/quickstart) after that I executed curl examples from documentation (https://wit.ai/docs/http/20160330#converse-link)

curl -XPOST 'https://api.wit.ai/converse?v=20160330&session_id=123abc&q=Whats%20the%20weather?' \
      -H "Content-Type: application/json" \
      -H "Accept: application/json" \
      -H 'Authorization: Bearer my_token'

{
  "confidence" : 1,
  "type" : "merge",
  "entities" : {
    "intent" : [ {
      "confidence" : 0.9793094394312485,
      "type" : "value",
      "value" : "atmosphere"
    } ]
  }
}

I always get good response from above example, but next one it's tricky

curl -XPOST 'https://api.wit.ai/converse?v=20160330&session_id=123abc' \
      -d '{"loc":"Warsaw"}' \
      -H "Content-Type: application/json" \
      -H "Accept: application/json" \
      -H 'Authorization: Bearer my_token'

{
  "confidence" : 0.02959246396722569,
  "type" : "msg",
  "msg" : "I will need API"
}

{
  "confidence" : 0.029646801991961923,
  "type" : "stop"
}

Some times i get question from bot (Where exactly?) and this is ok, but some times bot don't ask me that question and say another issue (I will need API) or stop conversation.

curl -XPOST 'https://api.wit.ai/converse?v=20160330&session_id=123abc' \
      -d '{"loc":"Warsaw","forecast":"sunny"}' \
      -H "Content-Type: application/json" \
      -H "Accept: application/json" \
      -H 'Authorization: Bearer my_token'

{
  "confidence" : 0.030503184791301343,
  "type" : "action",
  "action" : "fetch=forecast"
}

Maybe some one can explain how wit.ai stories works exactly, because that API it's unpredictable ;)

[darkin1]

How to use converse action like message action

    middleware.receive = function(bot, message, next) {
        if (message.text) {
            client.converse('my-user-session-42', message.text, {}, (error, data) => {
                if (error) {
                    console.log('Oops! Got an error: ' + error);
                    next(error);
                } else {

                    message.entities = data.entities;
                    next();
                    // console.log('value', data.entities.intent[0].value);
                    // console.log('confidence', data.entities.intent[0].confidence);
                    // console.log('intent',data.entities.intent);
                    // console.log('data',data.entities);
                    // console.log('Yay, got Wit.ai response: ' + JSON.stringify(data));
                }
            });

        } else {
            next();
        }
    };

    middleware.hears = function(tests, message) {

        if (message.entities) {
            for (var i = 0; i < message.entities.intent.length; i++) {
                for (var t = 0; t < tests.length; t++) {
                    if (message.entities.intent[i].value == tests[t]) {
                        return true;
                    }
                }
            }
        }

        return false;
    };

[dfischer]

Yeah but doesn't the converse action have to loop in order to give you entities the way wit.ai expects? I.e merge/say stages?

I.e hit it once to see if it's merge. Hit it again to see if it's something else? https://wit.ai/docs/http/20160330#converse-link

"You need to call converse until the api says stop"

---

It is very confusing actually. I think I'm going to just switch to http://www.api.ai

I'm also very disappointed with the help on wit. I've waited for a few days for answers, and over 24 hours re: broken api endpoints.

They're just too early to support at the moment I'd say.

[robguilfoyle]

@dfischer I too am bummed with wit.ai's responses and the fact they are deprecating the intent API as quickly as they are.

[dfischer]

Yeah, this kinda rubbed me the wrong way. If there API/service is this experimental it would make sense for them to state this on their site. Not only is this a devastating production level change (broke API) but it wasn't replied to via support or even updated in their github until 1 hour ago of this comment (4 days time). https://github.com/wit-ai/node-wit/commit/6c1b4eb7d23a51577a705cb882f81e804d3fb245

I really want to love wit.ai. Rooting for them 👍 - just please be more vocal/warning next time!

[prototypeverything]

Hi Guys, I've been following your discussion, since this change threw me off as well...

My workaround (for now) is to just Fork an older Wit.ai project, so I can still build with intents:

https://wit.ai/mbrevoort/witbot-sample

I know this is not future proof, but it will give you something to work with in the mean time. And with the speed they switched Intents off, maybe they will bring them back as well?

[robguilfoyle]

@guusbaggermans: the API responses on their legacy projects are coming back with:

'Deprecated'

In the payload, it appears they will be shutting off those end points at some point.

[prototypeverything]

Correct @robguilfoyle, but since there is no proper way right now to work together with their current release, I guess it's a good a workaround as any.

[dfischer]

They released updated API end points that don't show deprecated

[robguilfoyle]

@dfischer: Have any info on those? Is it still the intent endpoint?

[dfischer]

@robguilfoyle https://github.com/wit-ai/node-wit/commit/6c1b4eb7d23a51577a705cb882f81e804d3fb245

[dfischer]

Playing around again with this now that the dust has settled.

I'm still curious how to implement the converse api but seems like a lot of overlap with the point of howdy. Need to figure out the right abstraction for looping over. @darkin1 you ever play more with the converse api and trying to loop over until it's finalized?

[darkin1]

Nope, I now testing api.ai platform. It's simpler and API it's more stable ;)