Handshake fails on iOS/Android

[ayyoitsp]

Handshake is failing due to mismatched auth. It may be an endian issue. Lost my output, but will attach some logs tomorrow.

[ayyoitsp]

Here's the log message that is suspicious..

Websocket Handshake failed: Invalid Sec-Websocket-Accept (expected I6DIugAAAAAAAAAAMDwGHQAAAAA= got I6DIut0xftuaMRqJMDwGHa1OlX0=)

Notice that the hashes more or less match, except the expected hash has many 'A's. It seems like the result of the sha1 has a chunk of zeros where it shouldn't be.

[ayyoitsp]

Also, this is Lua 5.1.4.

It works fine on MacOSX, but on iOS and Android devices, it does not.

[lipp]

@ayyoitsp : Thanks for reporting! Maybe you could provide the "Sec-Websocket-Key" HTTP Header (by wireshark tracing). This would be just awesome. Are this big endian machines?

Out of curiosity: Can you tell me something about your app / dev environment?

[ayyoitsp]

I don't have wireshark at the moment and I won't be back on the project for another few days. If that piece is critical, I can talk to the server engineer to get me a dump then.

My Mac (where it works fine) is big endian, the ARM iOS device is little endian. The app runs on iOS/Android/MacOSX. It runs under moai, which is a cross platform lua based platform. Let me know if you need anything else for now..

[BoarK]

ayyoitsp,

Are you using the ev client? I'm getting the same error message running the sync client on Corona SDK. With Corona I can't load my own C modules (without enterprise subscription) so I resorted to using numberlua in place of luabitop and hacking something together to replace lpack. I guess moai does not have this limitation.

The client is on a Mac osx 10.7.5 connecting to the server at ws://echo.websocket.org. I think both should be little-endian.

lipp: Could it be an endian issue?

[lipp]

@BoarK : I am not quite sure yet about the endianess... The relevant lpack encoding / decoding uses '>' and '<' directives to be explicit about byte order. lua-websocket (ev-server) is running on a 32bit big endian powerpc (at my work). The busted test system is not running on the (embedded) powerpc, so I can't tell about all busted test (it#s on TODO), but in generell it is working. Maybe you guys could add some prints to the websocket.tools.sha1. Maybe there is some variable overflow issue.....

[lipp]

@BoarK: If there are lpack alternatives, I can add support for them.

[BoarK]

lipp,

I replaced the sha1 implementation with one from Corona and got a match on the sec-websocket-accept. It turns out that my lpack replacement isn't working properly.

There is vstruct but it requires loadstring() which is disabled in Corona for security reasons. I am working off the write_format and read_format examples on lua-users.org:

http://lua-users.org/wiki/ReadWriteFormat

Can you tell me if this is correct? b = 1 byte, H = 2 bytes, I = 4 bytes?

[lipp]

Yes, this is the intention. Capital letter means signed, whereas Uncap is unsigned. This might be actually the problem, since the type width is platform dependent...

from lpack.c (source):

define OPZSTRING 'z' /* zero-terminated string /

define OPBSTRING 'p' / string preceded by length byte _/

define OPWSTRING 'P' / string preceded by length word _/

define OPSSTRING 'a' / string preceded by length sizet /

define OPSTRING 'A' / string _/

define OPFLOAT 'f' / float _/

define OPDOUBLE 'd' / double _/

define OPNUMBER 'n' / Lua number _/

define OPCHAR 'c' / char _/

define OPBYTE 'b' / byte = unsigned char _/

define OPSHORT 'h' / short _/

define OPUSHORT 'H' / unsigned short _/

define OPINT 'i' / int _/

define OPUINT 'I' / unsigned int _/

define OPLONG 'l' / long _/

define OPULONG 'L' / unsigned long _/

define OPLITTLEENDIAN '<' / little endian _/

define OPBIGENDIAN '>' / big endian _/

define OPNATIVE '=' / native endian */

[BoarK]

lipp,

I installed lua 5.1.5 and lpack. I ran some tests and the results from my lpack replacement and the actual lpack match. So according to what you wrote above, the server at ws://echo.websocket.org might be on a different platform?

[BoarK]

lipp,

I just successfully got an echo back from ws://echo.websocket.org. It appears that the problem is in the Sha1 implementation.

The Sha1 implementation at this address is correct: http://regex.info/blog/lua/sha1 In the comments I found this : http://cube3d.de/uploads/Main/sha1.txt which is suppose to be 10x faster.

Also tested on Android device. Sent and received text block that's 2672 in length. Your implementation using big-endian to encode the data length works on both Macosx and Android. I failed to receive a response from the echo server when I tried little-endian.

[lipp]

@BoarK no, no, ... sorry. That's not what I wanted to say. The Websocket protocol is unaware of machine / platform . Any Websocket protocol compliant server must work with every protocol complient client.

What i meant is: The C types are not necessarily the same byte width on different platforms: E.g. C type "int" might be 4 bytes or on 64bit machine 8 bytes --> pack('>I',123) might return a (binary) string of length 4 or 8 depending on the platform / compiler / etc.

This might be an implementation issue of lua-websockets.

[lipp]

Great! Thank you! Could you be so kind to sum up what works and what does not. (some combinations of: OS,SHA algorithm,client/server,ev/copas/sync).

Further, could you be so kind and to run the websocket.tool.sha1 on 'The quick brown fox jumps over the lazy dog' with print patch added in SHA1 algorithm:

  print('A',h0,h1,h2,h3,h4)

  -- necessary on sizeof(int) == 32 machines
  h0 = band(h0,0xffffffff)
  h1 = band(h1,0xffffffff)
  h2 = band(h2,0xffffffff)
  h3 = band(h3,0xffffffff)
  h4 = band(h4,0xffffffff)

  print('B',h0,h1,h2,h3,h4)

  local result = spack('>I>I>I>I>I',h0,h1,h2,h3,h4)

  print('C',result:byte(1,#result))

  return result

[BoarK]

lipp,

Output of websocket.tools.sha1

On Macosx:

A 113239086628 106809667440 2626437021 2129182818 5818677219
B 1569936932 3730452336 2626437021 2129182818 1523709923
C 93 147 86 36 222 90 47 112 156 140 63 157 126 232 192 98 90 209 247 227

On Android:

A 113239086628 106809667440 2626437021 2129182818 5818677219 B 1569936932 3730452336 2626437021 2129182818 1523709923 C 93 147 86 36 222 90 47 112 156 140 63 157 126 232 192 98 90 209 247 227

OS: Macosx 10.7.5 (Corona simulator) SHA: Corona cryto lib CLIENT: sync SERVER: @ws://echo.websocket.org

OS: Nook tablet original running Android 4.1.2 SHA: Corona cryto lib CLIENT: sync SERVER: @ws://echo.websocket.org

[ayyoitsp]

I was using the sync client. I cannot easily get ev/copas running in my environment as it runs inside Moai.

[lipp]

Thanks for the usage info!

@BoarK are you sure about the sha1 output and input? My powerpc 32bit and linux i386 32bit give this output:

A 5097447878 10639714556 3984891617 -1149835463 4757646098 B 802480582 2049779964 -310075679 -1149835463 462678802 C 47 212 225 198 122 45 40 252 237 132 158 225 187 118 231 57 27 147 235 18

Maybe there is a typo (newline?) in the argument?

sha1('The quick brown fox jumps over the lazy dog')

[BoarK]

lipp,

Using Lua 5.1, lpack, numberlua compiled on Macosx 10.7.5 :

A 113239086628 106809667440 2626437021 2129182818 5818677219 B 1569936932 3730452336 2626437021 2129182818 1523709923 C 93 147 86 36 222 90 47 112 156 140 63 157 126 232 192 98 90 209 247 227

Using Lua 5.1, lpack, LuaBitOp compiled on Macosx 10.7.5 :

A 5097447878 10639714556 3984891617 -1149835463 4757646098 B 802480582 2049779964 -310075679 -1149835463 462678802 C 47 212 225 198 122 45 40 252 237 132 158 225 187 118 231 57 27 147 235 18

So it turns out to be numberlua that's making the output different.

https://github.com/davidm/lua-bit-numberlua

[BoarK]

lipp,

Sorry my bad. I required numberlua incorrectly. The my result does match with yours. This is running in Corona Simulator on Macosx 10.7.5 using numberlua and my own pack.lua

A 5097447878 10639714556 3984891617 -1149835463 4757646098
B 802480582 2049779964 -310075679 -1149835463 462678802
C 47 212 225 198 122 45 40 252 237 132 158 225 187 118 231 57 27 147 235 18

[lipp]

@BoarK Thanks. Ok, so in the corona sim, the websocket client sha1 works? What about the target (iPhone etc)? Does the sha1 always fail? If this occurs "random", could you please add a further print to the sha and post the result on error, please?

local sha1 = function(msg)
   print('M',msg)
   ....  

[BoarK]

lipp,

Output on xcode simulator:

13-04-29 12:39:30.135 PM Websocket: A 5097447878 10639714556 3984891617 -1149835463 4757646098 13-04-29 12:39:30.136 PM Websocket: B 802480582 2049779964 -310075679 -1149835463 462678802 13-04-29 12:39:30.137 PM Websocket: C 47 212 225 198 122 45 40 252 237 132 158 225 187 118 231 57 27 147 235 18

The output matches other platforms. The tools.sha1 does not fail in any of the tests.

Nathan.

[lipp]

@BoarK : I cannot figure out the picture of the situation.

• tools.sha1 seems to work on all targets (output on console is correct)
• handshake / accept still fails on iPhone and Android
• replacing tools.sha1 by crypto solves the problem

Is this correct?

[BoarK]

@lipp ,

Your sha1 implementation works fine. I narrowed it down to a problem with the luabitop replacement that I'm using, numberlua. At first I didn't require it properly so some of the bit operations were wrong causing the your sha1 function to fail. Then I ran into another problem with numberlua's implementation of bit-and (band) in frame.decode(). I replaced the band function with one from another source as a fix.

I can confirm that the client works fine on Corona simulator, xcode simulator (haven't tested on actual iOS device) and android 4.2.2.

[lipp]

@BoarK Thanks for investigating! How is the status? Are you fine with lua-websocket? Should I provide some functionality (choose between bitops?).

[lipp]

@ayyoitsp @BoarK Polling status :)

[BoarK]

@lipp,

I ported socket.io v0.9.11 client over to lua (websocket and xhr-polling only) and used lua-websocket as a base for the websocket component. Everything works fine. The only thing missing is SSL/TLS which Corona SDK does not yet support. Websocket support is essentially useless unless it is secure. I will have to wait on Corona to catch up. I wouldn't have been able to do this port without your excellent lua-websocket. Thank you.

[lipp]

@BoarK Pleasure is on my side! Is your socket.io port public? I will close this issue (so that it is not visible as bug any more). Regards

[BoarK]

@lipp,

No it's not public. I haven't tested it thoroughly yet. Plus it's using the timer functions and async network calls from Corona SDK. I threw together hacks as replacement for lpack and Bitop so that has to be tested. The good news is that Corona just added Bitop as a plugin. I am going to ask them to add lpack. Also, there is the matter of SSL/TLS support which the CEO of Corona said he would look into adding. There's quite a ways to go until it's release worthy. Thanks again.

[follop]

@BoarK

Would you share your version of lpack? I am completely stuck because corona can't use my C dlls.

[EnTerr]

Glad i found this issue, guys! I am trying the echo example to ws://echo.websocket.org with sync client (intel mac so far) and failing with "Websocket Handshake failed: Invalid Sec-Websocket-Accept (expected AnDKkAsrkWcWifKgdqIeekL8lH0= got gnDKkIsrkWcWifKg9qIeesL8lH0=)"

I am on pure Lua 5.1, so trying to use

• struct = lua-struct
• bit = lua-bit-numberlua

Thanks to the @lipp example ('The quick brown fox jumps over the lazy dog'), i was able to figure that i have to use numberlua in LuaBitOp compatiubility mode, i.e. bit = require 'bit.numberlua'.bit (and not in the default or bit32 compatibility mode). I suspect that's what @BoarK meant by "to require it properly'.

Now the puzzling thing is how come sha1 test comes correct but the error above occurs? Moreover the expected and actual values look so much alike, just like in the OP case?!

[EnTerr]

Update: turned out the issue was with the struct library doing some of the packing wrong. After fixing it websockets works like a charm!

So if someone else runs in this problem... check your struct functionality

[lipp]

@EnTerr :+1: what kind of struct problem did you have?

[EnTerr]

In my particular case, bytes were not getting packed/unpacked right https://github.com/iryont/lua-struct/issues/2