Websocket client gets unhandled IndexError exception

[hiteshjasani]

Nim Compiler Version 0.19.2 [MacOSX: amd64], Compiled at 2018-12-31 websocket [0.3.5]

If you run the following websocket client program you'll see that for any sleep time argument longer than 54000 (millis), it will get an IndexError at approx 54 seconds into the run.

Command Line	Result
time ./buggy_websocket 10000	As expected, runs for ~10 seconds
time ./buggy_websocket 20000	As expected, runs for ~20 seconds
time ./buggy_websocket 120000	Unexpectedly runs for only ~54 seconds
time ./buggy_websocket 240000	Unexpectedly runs for only ~54 seconds

Some thoughts:

• Is there a timeout or some setting that I'm missing that's causing the error at 54 seconds?
• For the times that the websocket is closed as expected, the library still throws an IndexError on websocket.close(). Is there a better way to close the socket without getting the error?

---

import asyncnet, asyncdispatch, os, websocket
from strutils import parseInt
from net import newContext, SSLContext, protSSLv23, CVerifyNone

let
  WS_HOST = "paper-api.alpaca.markets"
  WS_PATH = "/stream"
  WS_PORT = Port(443)
  WS_SSL = true

proc reader(ws: AsyncWebSocket) {.async.} =
  while true:
    let (opcode, data) = await ws.readData()
    echo "read -- opcode: ", $opcode, "  data: ", $data

proc pinger(ws: AsyncWebSocket) {.async.} =
  while true:
    await ws.sendPing()
    await sleepAsync(20_000)

when isMainModule:
  let sleepTimeMillis = parseInt(paramStr(1))
  echo "Sleep time is " & $sleepTimeMillis

  try:
    let sslCtx = newContext(protSSLv23, verifyMode = CVerifyNone)
    doAssert(not sslCtx.isNil, "Failed to initialize SSL context")
    let ws = waitFor newAsyncWebsocketClient(WS_HOST, WS_PORT,
                                             WS_PATH, WS_SSL,
                                             ctx = sslCtx)
    echo "Connected!"

    asyncCheck reader(ws)
    asyncCheck pinger(ws)

    waitFor sleepAsync(sleepTimeMillis)
    waitFor ws.close()

  except ProtocolError:
    echo ".... connection failed!"
    echo getCurrentExceptionMsg()
  except OSError:
    echo ".... connection failed!"
    echo getCurrentExceptionMsg()

---

Example stack trace

Sleep time is 240000                                                            
Connected!                                                                      
read -- opcode: Cont  data: 5c882fb62f85f47000000001                            
read -- opcode: Cont  data: 5c882fca2f85f47000000002                            
read -- opcode: Cont  data: 5c882fde2f85f47000000003                            
buggy_websocket.nim(36)  buggy_websocket                                        
asyncdispatch.nim(1656)  waitFor                                                
asyncdispatch.nim(1516)  poll                                                   
asyncdispatch.nim(1282)  runOnce                                                
asyncdispatch.nim(191)   processPendingCallbacks                                
asyncmacro.nim(36)       readData_continue                                      
shared.nim(136)          readDataIter
shared.nim(139)          makeFrame
shared.nim(134)          makeFrame
system.nim(2916)         sysFatal
[[reraised from:
buggy_websocket.nim(36)  buggy_websocket
asyncdispatch.nim(1656)  waitFor
asyncdispatch.nim(1516)  poll
asyncdispatch.nim(1282)  runOnce
asyncdispatch.nim(191)   processPendingCallbacks                               
asyncmacro.nim(36)       reader_continue
buggy_websocket.nim(13)  readerIter
asyncfutures.nim(302)    read
]]
[[reraised from:
buggy_websocket.nim(36)  buggy_websocket
asyncdispatch.nim(1656)  waitFor
asyncdispatch.nim(1516)  poll
asyncdispatch.nim(1282)  runOnce
asyncdispatch.nim(191)   processPendingCallbacks                               
asyncfutures.nim(349)    :anonymous
]]
Error: unhandled exception: index out of bounds                                
Async traceback:
  buggy_websocket.nim(36) buggy_websocket                                      
  asyncdispatch.nim(1656) waitFor
  asyncdispatch.nim(1516) poll
    ## Processes asynchronous completion events                                
  asyncdispatch.nim(1282) runOnce
  asyncdispatch.nim(191)  processPendingCallbacks                              
    ## Executes pending callbacks
  asyncmacro.nim(36)      readData_continue                                    
    ## Resumes an async procedure
  shared.nim(136)         readDataIter
  shared.nim(139)         makeFrame
  shared.nim(134)         makeFrame
  system.nim(2916)        sysFatal
  #[
    buggy_websocket.nim(36) buggy_websocket                                    
    asyncdispatch.nim(1656) waitFor
    asyncdispatch.nim(1516) poll
      ## Processes asynchronous completion events
    asyncdispatch.nim(1282) runOnce
    asyncdispatch.nim(191)  processPendingCallbacks
      ## Executes pending callbacks
    asyncmacro.nim(36)      reader_continue
      ## Resumes an async procedure
    buggy_websocket.nim(13) readerIter
    asyncfutures.nim(302)   read
  ]#
Exception message: index out of bounds
Exception type: [IndexError]

real    0m54.495s
user    0m0.029s
sys     0m0.013s

[dom96]

Try updating websockets to #head, I fixed this bug IIRC: nimble install websockets@#head

[hiteshjasani]

Same error. Here's what I did.

1. nimble install --verbose websocket@#head -- Note you have an extra s char in your command.
2. nimble list -i shows websocket [#head, 0.3.5] as installed.
3. Change the .nimble dependency section to requires "nim >= 0.19.0", "websocket#head"
4. time ./buggy_websocket 120000 results in the same IndexError after ~54 seconds.

[dom96]

Seems like a bug with our websocket client implementation. I tried looking into it but your code fails with ".... connection failed!" and a "server did not reply with a websocket upgrade:" exception. Not sure why that might be, any ideas?

[hiteshjasani]

@dom96 Those errors indicate the SSL connection didn't complete. I was actually getting that error myself and was able to bypass it by instantiating the SslContext myself. Try making the following change to see if this works for you.

Change this

    let sslCtx = newContext(protSSLv23, verifyMode = CVerifyNone)
    doAssert(not sslCtx.isNil, "Failed to initialize SSL context")
    let ws = waitFor newAsyncWebsocketClient(WS_HOST, WS_PORT,
                                             WS_PATH, WS_SSL,
                                             ctx = sslCtx)

to

    let ws = waitFor newAsyncWebsocketClient(WS_HOST, WS_PORT,
                                             WS_PATH, WS_SSL)

It'll use the default SslContext created in websocket/client.nim, which is protTLSv1.

BTW, can we reopen this issue? I'm still seeing the same behavior with the new code merged from #53.