Parse exception: not enough bytes

[nh2]

In an application using a websockets Haskell client that simply connects to a http server and prints to the terminal all messages sent by the server, I just got Parse exception: not enough bytes. (I cannot replay this, I suspect that the other server just crashed or restarted or so.)

I believe this comes from http://hackage.haskell.org/package/io-streams-1.1.0.3/docs/System-IO-Streams-Attoparsec.html#v:parserToInputStream.

I suspect that some of the messages were cut off in the middle or so. Anyway, it would be great if websockets could give me a more detailed error message, e.g. mentioning where some parse failed.

[ocharles]

I can reproduce this by connecting to a web socket and then simply closing the client. https://github.com/ocharles/snaplet-socketio/blob/5040b6fd43b9194b2d0005f1cf2289633d277f28/src/Snap/Snaplet/SocketIO.hs has this behaviour.

[blaenk]

I ran into this as well just now. I wanted a way to know if the client closed the connection, but when the client closes the connection it just crashes with this.

[jaspervdj]

So, an approach I could take is catching the ParseException and re-throwing it as a ConnectionClosed exception. Additionally, I'll have the connection keep an IORef Bool which indicates whether the connection was closed. Do you think this would be good for your use cases?

[nh2]

Do you think this would be good for your use cases?

It depends: Might there be other cases where we can clearly distinguish a ParseException (other side sent rubbish) from correct, but cut-off data?

Also, from my experience with other websocket stacks, it's always good to be able to distinguish a connection close by cut-off and a graceful connection close as defined by the protocol.

[jaspervdj]

Hmm, intuitively I don't think we want to distinguish between cut-off data and rubbish.

However, a graceful close and a cut-off is definitely something important. I've already been thinking to change the API to have

receive :: Connection -> IO (Maybe Message)

following the io-streams design where Nothing indicates clean EOF. On the other hand, this is another API-incompatible change and hence annoying for the users of this library.

[nh2]

Hmm, intuitively I don't think we want to distinguish between cut-off data and rubbish.

That does make sense, it's probably not even very possible.

On the other hand, this is another API-incompatible change and hence annoying for the users of this library

Following Api changes like this is easy in Haskell. Getting things run right is much harder, and any Api change that helps with that is a time saver for me. :)

[blaenk]

You know more about the websockets spec than I do. I looked it up and it seems like the peer just sends one byte to signify that it wants to close the connection. Is this why the parser is complaining that there aren't enough bytes to parse? Can't a new parser rule be added for this close-connection byte?

My confusion is why you guys are talking about it like it's cut-off or rubbish data when, from my admittedly superficial investigation, it seems like the client is actually sending meaningful data to the server, indicating that it wants to close the connection. That's why it's happening when the page is closed in the browser, for example.

Or is it perhaps that @nh2's case is indeed a matter of cut-off data? In that case, I think @ocharles and my case is different. In our case, the client is legitimately attempting to close the connection, but the websockets server doesn't understand the data the client sends to signify this, I guess (doesn't have a parser rule for it?), so it crashes trying to use that very short data to parse a regular message, which is why it's saying there aren't enough bytes to parse something out of? Just a guess.

[blaenk]

For what it's worth, the actual cut-off data scenario can be easily reproduced by clicking on a link to a page that establishes a connection to the WebSocket server, then immediately pressing the back button.

[jaspervdj]

Okay, I'm done with my christmas holidays and I'll be able to take some time to fix this properly. How I envision this will work is that ConnectionClosed will only be used in cases where the connection was cleanly closed. In other cases, there will be other appropriate exceptions.

This means that when writing WebSockets servers, it will be highly recommended to use combinators such as bracket and finally to wrap your handlers, instead of just catching ConnectionClosed.

[dmjio]

I'm getting this as well. The Parse exception: not enough bytes

I have browsers and backend servers connecting to my snap-server that is holding the websocket connections in an MVar (like from the example code). The backend workers will send messages to the webserver, which will look up browser connections and attempt to send a message to them.

@jaspervdj I've done as you've said and attempted to rethrow the the ParseException as a ConnectionClosed. I'm able to catch ParseException, but am unable to remove it from my MVar of clients. It seems I cannot access objects defined in a function scope outside my where clause. (I outline it in more detail below, probably just a dumb indentation error :)

Another problem I run into is when a user refreshes their browser in the same time period when a message is being sent to a client, this error is thrown. send: resource vanished (Broken pipe) -- on the backend client This happens intermittently and I can't seem to track down the exact cause, sometimes refreshing doesn't cause a problem. Uncertain what specific exception this is referring to and what library it comes from (Assuming io-streams or network). Any ideas? Just need to catch it somehow.

Browsers connect like :

var ws = WebSocket('wss://myaddress.com:443/websox/' + userLogin); 

Backend workers connect like:

 runClient "myaddress" 80 "/websox/worker/" $ \conn -> do
      sendTextData conn $ WorkerConnect ("worker" <> " " <> "worker1")

@jaspervdj, Another question, websocket clients (that aren't browsers) do not support wss, correct?

Here's my code that's trying to handle both the ParseException and the send: resource vanished (Broken pipe) code

application :: MVar ServerState -> WS.ServerApp
application state pending = do
    conn <-  WS.acceptRequest pending
    clients <- liftIO $ readMVar state
    cmd <- WS.receiveData conn :: IO Command
    case cmd of
      BrowserConnect email ->
          do acid <- openRemote
             e <- query acid $ ByLogin email
             forM_ e . const $ handle err $ addConnection email conn state
      WorkerConnect email -> handle err $ addConnection email conn state
      otherwise -> return ()
    where
      err :: ParseException -> IO a
      err (ParseException e) = do putStrLn "Caught one" >> putStrLn e >> throw WS.ConnectionClosed
                                  -- modifyMVar_ state $ return . removeClient (email, conn)

                                  -- I can catch the parse exception but
                                  -- can't add the above code because
                                  -- conn cannot be found... what are
                                  -- the scoping rules here? the
                                  -- `disconnect` function below seems
                                  -- to be accessing values defined in
                                  -- a higher scope from a where just fine...

addConnection :: Text -> WS.Connection -> MVar ServerState -> IO ()
addConnection email conn state =
    handle disconnect $
         do void $ liftIO $ modifyMVar_ state $ return . addClient (email, conn)
            st <- liftIO $ readMVar state
            liftIO $ print "added connection"
            print . map fst $ st
            forever $ talk conn state
          where
            disconnect :: WS.ConnectionException -> IO ()
            disconnect e = print e >> closer
            closer = modifyMVar_ state $ return . removeClient (email, conn)

[dmjio]

ended up just POST'ing json w/ http-conduit from the workers to the web server. I was probably doing something screwy above.

[davidxifeng]

how about change

-- file: Network/WebSockets/Hybi13.hs 
parseFrame :: A.Parser Frame

to

parseFrame :: A.Parser (Maybe Frame)

and use

-- System.IO.Streams.Internal.Attoparsec.hs
--If the parser yields Just x, then x will be passed along downstream, 
--and if the parser yields Nothing, that will be interpreted as end-of-stream.
parserToInputStream :: Parser (Maybe r) -> InputStream ByteString -> IO (InputStream r)

we get a InputStream Frame which Nothing is end of input.

bytestring read from websockets connection can be closed without friendly close control message

[davidxifeng]

and

decodeMessages :: Streams.InputStream ByteString
               -> IO (Streams.InputStream Message)
decodeMessages bsStream = do
    dmRef <- newIORef emptyDemultiplexState
    Streams.makeInputStream $ next dmRef
  where
    next dmRef = do
        frame <- Streams.parseFromStream parseFrame bsStream
        m     <- atomicModifyIORef dmRef $ \s -> swap $ demultiplex s frame
        maybe (next dmRef) (return . Just) m

can return Nothing when we could not Parse Frame in the received bytestring

[andreywix]

This issue is still present. Has anyone been able to find a solution which is not just accepting not enough bytes as connection close?

[jaspervdj]

This should be solved now @andreywix.

[ericpashman]

I've been seeing this exception recently in situations where there's no recent activity over the connection. Should I handle a ParseException as if it were a ConnectionClosed?

[domenkozar]

For anyone hitting this, I used https://hackage.haskell.org/package/retry to just retry connecting the client (that makes sense for my use case, but it might not work for every).

[domenkozar]

If someone can provide a reliable way to reproduce this, I'm willing to fix it.