handshake issue with utf-8 chars

aallen-dev commented 6 years ago

The client I am using is sending a Sec-WebSocket-Key header in a format incompatible with this lib, and I think it has to do with readStringUntil

According to Prashant Gurav there are (at least?) 2 handshake request types from clients. My experience seems to confirm this. Connecting from my computer on chrome "emulating" (agent spoofing) a mobile device everything works fine and using ESP8266WiFi.h and String line = client.readStringUntil('\n'); I show the following from the client

Connecting to wifi.................... connected!
Web server started 10.0.0.200
GET / HTTP/1.1
Host: 10.0.0.200
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_1 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko) Version/10.0 Mobile/14E304 Safari/602.1
Upgrade: websocket
Origin: http://10.0.0.93:3000
Sec-WebSocket-Version: 13
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Sec-WebSocket-Key: 1yiewrFWbfBZNYI3lruMlg==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits

but using my mobile phone I get a 400 response, and using the same test as above the following on the ESP8266 console output

Connecting to wifi.................... connected!
Web server started 10.0.0.200
GET / HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: 10.0.0.200
Origin: http://10.0.0.93:3000
Sec-WebSocket-Key1: 1 08Y     B  76966 38
Sec-WebSocket-Key2: 22> 25R7 5e8 D*0P g9 8 O

Uϓ�V-�

This looks like the second type and I'm assuming it is valid for the spec. My first thought was that readStringUntil wasn't parsing the string as utf-8, but the answer here, on the same subject, implies that it would...

I still think there is something fishy in readStringUntil because of the output in the second test. not only is there a blank line before the unknown characters, but that blank line also represented a long hang time.

Links2004 commented 6 years ago

wow you found a strange client implementation. in what browser / phone / lib did you find it?

the Sec-WebSocket-Key1, Sec-WebSocket-Key2 stuff never left the draft state and is expired on March 28, 2011. https://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-02#section-9.4

this library implements RFC6455 https://tools.ietf.org/html/rfc6455#section-11.3.1 which only uses Sec-WebSocket-Key as base64 encoded value.

Prashant Gurav does mix different the RFC6455 and a old draft version. the RFC6455 is the standard to use for websocket all other stuff is old draft garbage.

all Browsers support the RFC6455 https://developer.mozilla.org/de/docs/WebSockets

hope you can use a RFC6455 conform websocket on you mobil phone.

aallen-dev commented 6 years ago

gotcha.

It's an old iphone. I looked it up and it uses ios 5.1 which seems to have been from 2011... I checked my ipad (ios 9.3.5) and it works just fine.

As far as the lib/browser, I think they just used the os level websockets implementation, because both safari and chrome are doing the same thing.

So, I'm trying to put together some home automation stuff that uses old tablets/phones as control panels. it simplifies mounting, offloads some processing power to the more powerful phone cpu and is way cheaper. I can get a used smartphone/tablet circa 2010 for <$20 vs an arduino compatible touch display (of comparable size) for >$50.

I worry that it might be hard for novices to get their hands on compatible os's and especially to jailbreak old iphones etc.

I'm trying to make it as backwards compatible as possible to keep it simple. I'm using a build system that I made a while back that transpiles es2015 down to es3 for working in classic asp/jscript.net/hta so that I can be 100% sure any device can use the same codebase.

for now it just falls back to REST/long polling if websockets are not available, but in this case it said websockets were available, then it just failed.

aallen-dev commented 6 years ago

it's obviously out of this projects scope to support an old non ratified spec, but is there any way you can assist me in making a fork with this capability? I'm sure I can handle at least that if I understood the handshake a little better. Assuming their are no more discrepancies beyond the handshake...

Links2004 commented 6 years ago

after a fast overlook over the old draft, it looks like its using the some similar data encoding and "opcode" like the RFC6455. so if the header HTTP handshake is adapted you also need to adapt the opcode part

draft:

  frame-opcode           = %x0 ; continuation frame
                             / %x1 ; connection close
                             / %x2 ; ping
                             / %x3 ; pong
                             / %x4 ; text frame
                             / %x5 ; binary frame
                             / %x6-F ; reserved

RFC6455:


     |Opcode  | Meaning                             | Reference |
    -+--------+-------------------------------------+-----------|
     | 0      | Continuation Frame                  | RFC 6455  |
    -+--------+-------------------------------------+-----------|
     | 1      | Text Frame                          | RFC 6455  |
    -+--------+-------------------------------------+-----------|
     | 2      | Binary Frame                        | RFC 6455  |
    -+--------+-------------------------------------+-----------|
     | 8      | Connection Close Frame              | RFC 6455  |
    -+--------+-------------------------------------+-----------|
     | 9      | Ping Frame                          | RFC 6455  |
    -+--------+-------------------------------------+-----------|
     | 10     | Pong Frame                          | RFC 6455  |
    -+--------+-------------------------------------+-----------|

may the protocol is different on binary level too, but i not checked it.

aallen-dev commented 6 years ago

ok, looks like I'm gonna need to circle back to this after some research. I was hoping it wouldn't be so low level as that... Coming from js a lot of c++ is pretty straight forward, but a lot is still pretty hard, and some concepts just never come up in js... I'm going to have to familiarize myself with "opcode" and the rest of your source code.

the more I think about it, the more I think it's worthwhile. I tested that phone on a lot of sites and with .net and nodejs servers and everything else works for websockets. With so many server implementations supporting it and considering it was once backed by apple I'm thinking this might not be the last time I run into it. Given that I tend to work with a lot of legacy systems, it would be real nice to have this solved and in my back pocket :)

Thank you so much for your help, quick response time, and of course this lib. Love your work btw

Links2004 / arduinoWebSockets

handshake issue with utf-8 chars #380