I'm getting the wrong data after the update

[TopherTheGopher]

Do you want to request a feature, report a bug, or ask a question about wit?

bug/question

What is the current behavior?

So the problem appeared after the update on the 28th. Before the update I would be able to pull a json file looking like this:

{ "entities": {}, "intents": [], "text": "why you no work.", "traits": {} }

I'd be able to take the text from here and have it do whatever commands I wanted in my Node js program.

Now when I pull the json file I get this:

{ "text": "why" } { "text": "why you", } { "text": "why you no", } { "text": "why you no work", } { "entities": {}, "intents": [], "text": "why you no work.", "traits": {} }

I have no idea how to stop getting the script breakdown and just get the final chunk of data.

This is the code in Node js I'm running to get the json file:

    const witClient = require('node-witai-speech');
    const extractSpeechIntent = util.promisify(witClient.extractSpeechIntent);    
    var stream = Readable.from(buffer);
    const contenttype = "audio/raw;encoding=signed-integer;bits=16;rate=48k;endian=little"
    const output = await extractSpeechIntent(WITAPIKEY, stream, contenttype)
    console.log(output)

    if (output && '_text' in output && output._text.length)
        return output._text
    if (output && 'text' in output && output.text.length)
        return output.text
    return output;

I'm at a loss in trying to find what I need to change to pull the correct data since I don't know what Wit AI changed on their end.

[francescor93]

Hi @TopherTheGopher , i am experiencing the same problem in my app written in PHP. According to the documentation (https://wit.ai/docs/http/20210928/#post__speech_link), it appears that the /speech endpoint now returns partial transcripts as well, and only the latter is the full object that interests us, unlike the endpoint /message which only returns the full object. The difficulty in obtaining the results we want derives from the fact that the response is not an array of objects, from which it would be sufficient to extract the last one, but simply a list not enclosed in anything, so it is not considered a valid json by the programming language (in my case, the _jsondecode() function I used until now returns null). I am following this issue because I hope to have more information about it; I don't know why the wit.ai team introduced this change, and I look forward to their offering us the option to disable this behavior, or at least to enclose the result in an array, so that we can handle it more effectively.

In the meantime I have applied a change to my code, which I report here because I hope it can suggest a solution: Before I json_decoded the response as it was; but now I go through a few more steps

• $objects = preg_split("/(^{|}\s*{|}$)/", $json);: Since we get a list of objects, let's separate them at the beginning (the ^{ part ), between one and the other (the }\s*{ part) and at the end (the }$ part), thus obtaining an array containing the individual objects
• array_pop($objects);: Let's remove the last element of the array since, due to our separation, it will always be empty
• $json = end($objects);: Now, from the remaining array, let's get the last element, which will be the full object we wanted
• $json = "{" . $json . "}";: Let's enclose the resulting string between { and } to obtain a valid json object
• Then let's decode the json and proceed as usual

I've tested this method with both the endpoint /speech response and the endpoint /message response and it seems to work fine in both cases. Again, I look forward to the development team helping us handle this behavior on their side as well.

Regards, Francesco

[n0th1ng-else]

I am facing the same in my NodeJS app as well the issue is also that the response does not contain the transfer-encoding header so we can understand what has been changed lately

[n0th1ng-else]

I made a hacky solution to resolve the issue (simplified code snippet)

....post()
      .then(response => {
        try {
            const data = JSON.parse(response); // <<-- If parsed well there was a single chunk and all is good
            return data;
        } catch (e) {
            const chunks = response.split('\r\n'); // <<-- Split into chunks array
            const lastChunk = chunks.pop(); // <<-- get the last chunk
            const data = JSON.parse(lastChunk); // <<-- parse last chunk
            return data;
        }
      })

[patapizza]

Hi,

Since 20210928 we've added live transcription support to our POST /speech API. Splitting HTTP chunks with \r\n is the correct approach. Here's an example integration in Node.js: https://github.com/wit-ai/node-wit/blob/main/lib/wit.js#L154.

Hope this helps.