100-continue and PUT request - they hang on env['wsgi.input'].read() call

[jn0]

wsgiserver 1.3 in Python 2.7.12 under Ubuntu 16.04

Create the script

#!/usr/bin/python

import sys
import wsgiserver
import pprint
import signal

sample_text = [
    'Hello world\n',
    '\n',
]

def The_Application(env, proc):
    status, response_headers, result = \
        '200 OK', [('Content-Type', 'text/plain; charset=utf-8'),], []

    # cardinal workaround: don't allow a thread to hang the node forever
    signal.alarm(15)

    pprint.pprint(env)
    pprint.pprint(server)

    data = env['wsgi.input'].read() # XXX this will block forever XXX
    pprint.pprint(data)

    # poor attempt to work it around
#   if env.get('HTTP_EXPECT', '-') == '100-continue':
#       proc('100 Continue', [])

    for x in sample_text:
        result.append(x)

    proc(status, response_headers)
    return result and result or ['ERROR']

wsgi_pathes = {
    '/': The_Application,
}

dispatcher = wsgiserver.WSGIPathInfoDispatcher(wsgi_pathes)

server = wsgiserver.WSGIServer(dispatcher, host = '0.0.0.0', port = 8080)

try:
    server.start()
except e:
    server.interrupt = e
    server.stop()
    sys.stderr.write('Killed\n')

# EOF #

run it and call with curl. You'll see that

curl -vv -F a=b http://127.0.0.1:8080/some/path/to

performs just fine with POST method, but

curl -vv -F a=b -T some.small.file http://127.0.0.1:8080/some/path/to

will cause the app (and the whole box!) to hang until alarm just after sending HTTP/1.1 100 Continue response.

If one comment out the .read() call, then normal operation resumes:

> PUT /some/path/to HTTP/1.1
> Host: 172.17.16.105:8080
> User-Agent: curl/7.47.0
> Accept: */*
> Transfer-Encoding: chunked
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
< Date: Tue, 06 Dec 2016 14:37:52 GMT
< Server: jno-intra
< 
Hello world

Here we are.

---

I.e. it's the .read() method of ChunkedRFile class from wsgiserver.py module.

If the size parameter is not set, then it could set once the self.buffer to something and loop forever eating all the RAM available (in data += self.buffer).

I think, the line # 457

                data += self.buffer

should read somehow as

                data += self.buffer; self.buffer = None

---

Yes, it works for me with this "patch".

[jn0]

BTW, line # 483 in .readline() method has also such an "unguarded" data update which may persist forever if self.buffer has no LF...

[jn0]

Plus, it may be a good idea to

1. make a worker thread to call self.setDaemon(True) in .__init__() to get killed along with the main thread
2. make'em "stoppable" for graceful shutdown.
3. provide a mean to use with-as syntax (i.e. add .__enter__() and .__exit__() methods)

I'd like to call it as

with WSGIServer(...) as server:
    server.start()
    ###

[fgallaire]

@jn0 As WSGIserver is a standalone fork of the CherryPy WSGI server, could it be possible for you to test it too ?

[fgallaire]

seems related to https://github.com/cherrypy/cherrypy/issues/1486

[jn0]

Yes, it looks the same. And needs almost the same patch.

[webknjaz]

@jn0 This is fixed in cherrypy/cheroot upstream. Please use cheroot of version v5.6.0