uvLoop/Protocol Error? - Githubissues

imgurbot12 commented 7 years ago

I was testing a server using Japronto in an Ubuntu-14.04 64bit Virtualbox VM when i came across this error:

Warning: Host CPU doesn't support SSE 4.2, selecting slower implementation Accepting connections on http://0.0.0.0:80 protocol.pause_writing() failed protocol: <cprotocol.Protocol object at 0x240f7f0> transport: <TCPTransport closed=False reading=True 0x233ac38> Traceback (most recent call last): File "uvloop/handles/basetransport.pyx", line 92, in uvloop.loop.UVBaseTransport._maybe_pause_protocol (uvloop/loop.c:64797) AttributeError: 'cprotocol.Protocol' object has no attribute 'pause_writing' protocol.resume_writing() failed protocol: <cprotocol.Protocol object at 0x240f7f0> transport: <TCPTransport closed=False reading=True 0x233ac38> Traceback (most recent call last): File "uvloop/handles/basetransport.pyx", line 108, in uvloop.loop.UVBaseTransport._maybe_resume_protocol (uvloop/loop.c:65168) AttributeError: 'cprotocol.Protocol' object has no attribute 'resume_writing'

here is the source code that caused the error. It happens every time i load the page "/register"

from base64 import b64encode
from datetime import datetime
from japronto import Application, RouteNotFoundException

#***Variables***#
with open("images/img.jpg", "rb") as f: imgdata = b64encode(f.read()).decode("utf-8")
basehtml = """
<!DOCTYPE html>
<html>
<head>
    <title>Registration</title>
    <style>
        html {{
            background: url("data:image/png;base64,%s") no-repeat center center fixed;
            -webkit-background-size: cover;
            -moz-background-size: cover;
            -o-background-size: cover;
            background-size: cover;
        }}
        .center {{
            width: 310px;
            height: 300px;
            position: absolute;
            top:0;
            bottom: 0;
            left: 0;
            right: 0;
            margin: auto;
        }}
    </style>
</head>
<body>
<form id="register" method="POST" autocomplete="off">
    <div class="center">
        <div id="formobject" style="width:310px">
            <h3 align="center" style="color:rgb(255, 255, 255)">Registration</h3>
            <input name="firstname" required="required" pattern="[A-Za-z0-9]{{1,20}}" type="text" style="width:300px" placeholder="first-name">
            <br>
            <input name="lastname" required="required" pattern="[A-Za-z0-9]{{1,20}}" type="text" style="width:300px" placeholder="last-name">
            <br>
            <input name="email" required="required" pattern="[^@]+@[^@]+\.[a-zA-Z]{{2,6}}" type="text" style="width:300px" placeholder="email@here.com">
            <br><br>
            <button style="float:right">Submit</button>
        </div>
        <div id="noticeobject">{notice}</div>
    </div>
</form>
</body>
</html>
"""%(imgdata)

#***Functions***#
def page(request):
    html = basehtml.format(notice="")
    return request.Response(text=html, headers=server_headers(), mime_type="text/html")

def server_headers():
    return {
        "Server": "nginx",
        "Date": datetime.now().strftime("%a, %d %Y %H:%M:%S GMT"),
        "Last Modified": "Sun, 19 2017 09:00:01 GMT",
        "Accept Ranges": "bytes",
    }

def handle_404(request, exception):
    """Handles any 404 errors within Server"""
    print("[404] FROM: %s, TO: %s"%(request.remote_addr,request.path))
    """Handles 404 Requests"""
    html="""<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
"""
    return request.Response(text=html, headers=server_headers(), code=404, mime_type="text/html")

#***Classes***#
class NewApp(Application):
    """Used to modify the error handler to provide a better response on server and client"""
    def protocol_error_handler(self, error):
        """System Error Handler"""
        if error == "malformed_headers": error+=" (Possible Port Scan or Attack)"
        print("System Error: %s"%(error))
        error = error.encode('utf-8')
        response = """<html>
<head><title>400 Bad Request</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<hr><center>nginx</center>
</body>
</html>
"""
        return response.encode('utf-8')

#***Start***#
app = NewApp()
app.router.add_route('/register', page)
app.add_error_handler(RouteNotFoundException, handle_404)
app.run(port=80,debug=False)

This error does not occur when i remove the base64 encoded image from the html, is uvloop having trouble writing all the data at once?

squeaky-pl commented 7 years ago

This is probably happening because asyncio protocols are supposed to implement pause_writing and resume_writing whenever underlying buffers get full but in didn't get to this.

imgurbot12 commented 7 years ago

Ok, is there any way i can fix this? Or is this a module issue?

squeaky-pl commented 7 years ago

The correct way to fix this would be implementing control flow callbacks (pause_writing and resume_writing) and not flushing entire buffer if it's higher than transport's high water mark.

For now what you can try doing is putting request.transport.set_write_buffer_limits(high=<a value enough to fit whole response>). You are gonna need to play with this value a little to find one that let's underlying transport drain data fast enough and not overflow. https://docs.python.org/3/library/asyncio-protocol.html#asyncio.WriteTransport.set_write_buffer_limits

azzuwan commented 7 years ago

@imgurbot12 did setting the set write buffer limit worked for you? I have put it as high as 16MB it is still trying to pause and resume the buffer.

UPDATE: The high limit according to request.transport.get_write_buffer_limits() is already 65536 by default. I actually set it lower. Still get the same behavior even when it is set higher than that.

bawr commented 6 years ago

@squeaky-pl - If you'd give me some pointers on what these methods should actually do, I wouldn't mind implementing them, Python's C API doesn't scare me anymore. (We're happily using Japronto on an internal production system, and the workaround works, but is obviously not ideal.)

squeaky-pl / japronto

uvLoop/Protocol Error? #76