http_parser takes one byte to much, upgrade websockets

[quazzie]

• Version: v9.2.0
• Platform: Windows 64bit
• Subsystem: http_parser

http_parser takes one byte to much from this: `HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: upgrade Transfer-Encoding: chunked Sec-Websocket-Accept: aFYFO80e6Cf2ijYLvl+mWNmnXtg= Sec-Websocket-Extensions: permessage-deflate Content-Type: application/octet-stream Date: Wed, 20 Dec 2017 14:17:57 GMT Server: Python/3.6 aiohttp/2.3.5 EndTime: 15:17:58.104 ReceivedBytes: 0 SentBytes: 0

Á+ªV©,HU²RPJ,-ɈÏÏVÒQPÊHŒ/K-ÎÌÏIè™Zê)Õ `

Base64 of data: SFRUUC8xLjEgMTAxIFN3aXRjaGluZyBQcm90b2NvbHMNClVwZ3JhZGU6IHdlYnNvY2tldA0KQ29ubmVjdGlvbjogdXBncmFkZQ0KVHJhbnNmZXItRW5jb2Rpbmc6IGNodW5rZWQNClNlYy1XZWJzb2NrZXQtQWNjZXB0OiBhRllGTzgwZTZDZjJpallMdmwrbVdObW5YdGc9DQpTZWMtV2Vic29ja2V0LUV4dGVuc2lvbnM6IHBlcm1lc3NhZ2UtZGVmbGF0ZQ0KQ29udGVudC1UeXBlOiBhcHBsaWNhdGlvbi9vY3RldC1zdHJlYW0NCkRhdGU6IFdlZCwgMjAgRGVjIDIwMTcgMTQ6MTc6NTcgR01UDQpTZXJ2ZXI6IFB5dGhvbi8zLjYgYWlvaHR0cC8yLjMuNQ0KRW5kVGltZTogMTU6MTc6NTguMTA0DQpSZWNlaXZlZEJ5dGVzOiAwDQpTZW50Qnl0ZXM6IDANCg0KwSuqViqpLEhVslJQSiwtyYjPz1bSUVDKSIwvSy0qzszPA0kY6Jla6hkp1QIA

The upgraded socket connection is missing Á in its buffer. Works if i use npm: http-parser-js.

[bnoordhuis]

Can you post steps to reproduce?

[quazzie]

Hmm not really without setting up a whole system..

Server is homeassistant (http://hass.io).

Code to trigger error in node:

const WebSocket = require('ws');
const conn = new WebSocket('ws://homeassistantip:8123/api/websocket');

Tried to debug the internals of node but only got to socketOnData in _http_client.js There i saw that parser.execute returned a length that takes an extra byte of the data.

[quazzie]

Maybe if one instantiated a http_parser and tried to parse the data i posted it could return the same error.. i'll have to check on that tomorrow.

[bnoordhuis]

A simple test case that starts a server with http.createServer() and sends a raw request with net.connect() and socket.write() is perfectly acceptable. As long as it fails the same way on your and my machine.

[quazzie]

Testcase: need is the data i was expecting back after the upgrade. But the data i get is missing the first websocket framing byte (contains FIN and OPCODE).

const http = require('http');

const response = new Buffer("SFRUUC8xLjEgMTAxIFN3aXRjaGluZyBQcm90b2NvbHMNClVwZ3JhZGU6IHdlYnNvY2tldA0KQ29ubmVjdGlvbjogdXBncmFkZQ0KVHJhbnNmZXItRW5jb2Rpbmc6IGNodW5rZWQNClNlYy1XZWJzb2NrZXQtQWNjZXB0OiBQelViSG45N2lFcHcrczZmc1hGcVMvYUlyQjg9DQpDb250ZW50LVR5cGU6IGFwcGxpY2F0aW9uL29jdGV0LXN0cmVhbQ0KRGF0ZTogV2VkLCAyMCBEZWMgMjAxNyAxMzo1ODo1MyBHTVQNClNlcnZlcjogUHl0aG9uLzMuNiBhaW9odHRwLzIuMy41DQpFbmRUaW1lOiAxNDo1ODo1Ni44NzUNClJlY2VpdmVkQnl0ZXM6IDANClNlbnRCeXRlczogMA0KDQqBK3sidHlwZSI6ICJhdXRoX29rIiwgImhhX3ZlcnNpb24iOiAiMC41OS4yIn0=", "base64");
const need = new Buffer("gSt7InR5cGUiOiAiYXV0aF9vayIsICJoYV92ZXJzaW9uIjogIjAuNTkuMiJ9", "base64");

const server = http.createServer(function(req, res) {
  res.writeHead(200, { 'Content-Type': 'text/plain' });
  res.end('okay');
});
server.on('upgrade', (req, socket, head) => {
  socket.write(response);
  socket.pipe(socket); // echo back
});
server.listen(7000, () => {
  const req = http.request({ 
    hostname: 'localhost', 
    port: 7000, 
    headers: { 
      'Sec-WebSocket-Version': 13,
      'Sec-WebSocket-Key': '8ABg8JJUAN8w8NtnY1A1XA==',
      'Connection': 'Upgrade',
      'Upgrade': 'websocket',
      'Host': 'localhost:7000'
    } 
  });
  req.end();

  req.on('upgrade', (res, socket, upgradeHead) => {
    console.assert(upgradeHead.length === need.length, 'Length mismatch');
  });
});

[quazzie]

Or need can even be a slice of response for better clarity maybe. const need = response.slice(-45);

[bnoordhuis]

Thanks, I can reproduce and a fix is on the way.

I'm surprised this bug managed to stay hidden for so long. I thought at first it must be a recent regression but it seems to have been around for some time.

[Fishrock123]

I guess people don't often send body data in upgrade requests? Good catch either way.

@quazzie thanks for coming up with a testcase!

[lpinca]

I guess people don't often send body data in upgrade requests? Good catch either way.

This, RFC 6455 (WebSocket) says that data transfer should start only after a successful opening handshake.