wahern / cqueues

Continuation Queues: Embeddable asynchronous networking, threading, and notification framework for Lua on Unix.
http://25thandclement.com/~william/projects/cqueues.html
MIT License
244 stars 37 forks source link

exceeded unchecked error limit (Broken pipe) #210

Open Mehgugs opened 5 years ago

Mehgugs commented 5 years ago

I think this is a problem on my end just wanted to know what this error actually means.

/usr/local/share/lua/5.3/cqueues/socket.lua:578: exceeded unchecked error limit (Broken pipe)
Lua 5.3.5  Copyright (C) 1994-2018 Lua.org, PUC-Rio
> require"cqueues".VERSION
20171014

I believe this error occurs when calling

function send(state, op, d, identify)
    state.shard_mutex:lock()
    local success, err
    if identify or state.session_id then
        if state.connected then
            success, err = state.socket:send(encode {op = op, d = d}, 0x1)
        else
            success, err = false, 'Not connected to gateway'
        end
    else
        success, err = false, 'Invalid session'
    end
    state.shard_mutex:unlockAfter(GATEWAY_DELAY)
    return state, success, err
end

state.socket is a lua-http websocket client.

Only thing possibly worth noting is that I have interposed wrap to provide some debug logging.

Mehgugs commented 5 years ago

Okay so after reading up in the manual, I appear to be getting too many EPIPEs. I dont think this is my fault, since I'm only calling send here (in the case that caused the error) in response to a heartbeat message from socket so I should be connected, (my logs show no close message was sent to the websocket so in theory I should be able to send fine)

In the manual it mentions clearing errors, but I'm not sure exactly what it means by this. Should I just be handling EPIPE then calling clearerr ?

daurnimator commented 5 years ago

If you log any errors returned by socket functions, what does EPIPE come from?

Mehgugs commented 5 years ago

I'm not sure exactly because there's some indirection (i need to put an xpcall inside wrap to get the inner stack of the function which received the thrown error) but i believe it's state.socket:send which is returning EPIPE. This is likely a case of me not handling these correctly (I dont know how to clear the errors so the count will only increase over time) and the read direction works correctly since I still receive events from the server.

pspacek commented 5 years ago

FYI we have seen this on HTTP server built on lua-http:

pspacek commented 5 years ago
[worker.background] error: /usr/share/lua/5.1/cqueues/socket.lua:578: exceeded unchecked error limit (Broken pipe) stack traceback:
        [C]: in function 'send'
        /usr/share/lua/5.1/cqueues/socket.lua:578: in function 'xwrite'
        /usr/share/lua/5.1/http/h2_connection.lua:425: in function 'write_http2_frame'
        /usr/share/lua/5.1/http/h2_stream.lua:718: in function 'rst_stream'
        /usr/share/lua/5.1/http/h2_stream.lua:1199: in function 'shutdown'
        /usr/share/lua/5.1/http/server.lua:197: in function </usr/share/lua/5.1/http/server.lua:195>
pspacek commented 5 years ago

My guess is that EPIPE can happen if HTTP client terminates connection ungracefully or something like that.