Converting buffer toString() has issues with decodeURIComponent

[svarunan]

In Parser.js headText is the encoded String of Buffer. var headText = this.buffer.toString(this.encoding, 0, headEnd);

its trying to decode decodeURIComponent('TA-Sasivarunan%ED%A0%BD%ED%B1%8D-September%E2%80%932%E2%80%932016%20%E2%80%93%201.2/Dalvik/2.1.0%20(Linux%3B%20U%3B%20Android%205.0.1%3B%20GT-I9500%20Build/LRX22C)%20(belle-sip/1.4.1)')

Actual user agent string was 'TA-Sasivarunan👍-September–2–2016 – 1.2/Dalvik/2.1.0 (Linux; U; Android 5.0.1; GT-I9500 Build/LRX22C) (belle-sip/1.4.1)'

URIError: URI malformed
    at decodeURIComponent (native)
    at /mnt/node_modules/modesl/lib/esl/Parser.js:110:44
    at Array.reduce (native)
    at Parser._parseHeaderText (/mnt/node_modules/modesl/lib/esl/Parser.js:107:28)
    at Parser._parseHeaders (/mnt/node_modules/modesl/lib/esl/Parser.js:68:25)
    at Parser._onData (/mnt/node_modules/modesl/lib/esl/Parser.js:36:21)
    at Socket.emit (events.js:107:17)
    at readableAddChunk (_stream_readable.js:163:16)
    at Socket.Readable.push (_stream_readable.js:126:10)
    at TCP.onread (net.js:540:20)

For now i've update the code to

Parser.prototype._parseHeaderText = function(txt) {
    return txt.split('\n').reduce(function(headers, line) {
        var data = line.split(': '),
            key = data.shift(),
            value; 
        try{
            value = decodeURIComponent(data.join(': '));
        } catch(ex){
            console.log("ERROR DECODING URI: " + ex.message + " : "  + data);
        }
        if(key === 'Content-Length') {
            headers[key] = parseInt(value, 10);
        } else {
            headers[key] = value;
        }
        return headers;
    }, {});
};

Please shed some lights on now to prevent this error.

[stszap]

We encountered the same problem. It's very easy to reproduce.

nodejs script

var esl = require('modesl'),
conn = new esl.Connection('127.0.0.1', 8021, 'ClueCon', function() {
    conn.events('plain', 'CUSTOM')    
});

1. start freeswitch
2. start nodejs script
3. execute in fs_cli (192.168.88.254 is my local ip) originate sofia/internal/%DD@192.168.88.254 &echo()
4. nodejs crashes

/tmp/node_modules/modesl/lib/esl/Parser.js:110
            value = decodeURIComponent(data.join(': '));
                    ^

URIError: URI malformed
    at decodeURIComponent (native)
    at /tmp/node_modules/modesl/lib/esl/Parser.js:110:21
    at Array.reduce (native)
    at Parser._parseHeaderText (/tmp/node_modules/modesl/lib/esl/Parser.js:107:28)
    at Parser._parsePlainBody (/tmp/node_modules/modesl/lib/esl/Parser.js:154:24)
    at Parser._parseEvent (/tmp/node_modules/modesl/lib/esl/Parser.js:184:25)
    at Parser._parseBody (/tmp/node_modules/modesl/lib/esl/Parser.js:100:10)
    at Parser._parseHeaders (/tmp/node_modules/modesl/lib/esl/Parser.js:76:37)
    at Parser._onData (/tmp/node_modules/modesl/lib/esl/Parser.js:36:21)
    at emitOne (events.js:96:13)

[englercj]

Hmm, adding error handling for the parse failing seems like a good idea. But, it seems like the core issue here is that FSW is incorrectly Uri encoding components, no?

Specifically:

TA-Sasivarunan%ED%A0%BD%ED%B1%8D-

is not a valid Uri encoding:

encodeURIComponent('TA-Sasivarunan👍-September–2–2016 – 1.2/Dalvik/2.1.0 (Linux; U; Android 5.0.1; GT-I9500 Build/LRX22C) (belle-sip/1.4.1)');

// returns:
"TA-Sasivarunan%F0%9F%91%8D-September%E2%80%932%E2%80%932016%20%E2%80%93%201.2%2FDalvik%2F2.1.0%20(Linux%3B%20U%3B%20Android%205.0.1%3B%20GT-I9500%20Build%2FLRX22C)%20(belle-sip%2F1.4.1)"

They also aren't encoding the forward slashes ('/) with%2Fas they should. That shouldn't breakdecodeUriComponent` but it is still incorrect Uri encoding.

[svarunan]

I think its an issues on buffer.toString with utf-8 on Parser.js. Here is the place where it get's encoded.

[englercj]

@svarunan Buffer.toString() definitely does not URI encode stuff, FSW does that when it sends them over the wire. The protocol is text (utf8) and it sends items URI encoded so you can see the delimiters.

If the encoding is invalid, it is because of FSW. I would reach out to them and see if they can fix this. I had to fix their JSON encoding as well when I first made this library, so its likely this is probably fixable as well.

[englercj]

Going to close this for now since I am pretty sure this is an FSW issue, not a node-esl issue.