Accessing POST parameters in Proxy <request pyref=""> destroys them

[rbu]

Disclaimer: I've spent quite a while hunting down an obscure interaction between sockets, webob, deliverance and wsgiproxy. And I think deliverance is to blame, but I am not sure.

Given this xml config:

  <proxy path="/something">
...
    <request pyref="file://path/to/filter.py:munge" />
...
  </proxy>

and a filter.py like so:

def munge(request, log):
    request.params.get('fnord', None)
    return request

will consume all POST parameters in the request and the destination URL will not receive them.

Here's what happens:

1. A client sends a request to deliverance: "POST /something" with a form-encoded request body "foo=bar&fnord=no"
2. The dest filter reads the POST parameters
3. The request goes back out to the destination url
4. The destination URL will receive no content, i.e. no POST parameters

In particular, the problem is this:

1. The dest filter will receive the request
2. Accessing either POST or params will read() the request body (environ["wsgi.input"]) to generate the POST dictionary.
3. This read() is irreversible since socket's file-like object does not support seek().
4. The wsgiproxy / request.get_response() does not consider the POST variable dictionary and thus sends an empty input

To make this worse, bothrequest.copy() and Request(request.environ.copy()) do not help. The latter, which is also used in deliverance.proxy, just makes a shallow dict copy -- so we get the same file like object. The former makes an actual copy of the file, however the original request is thus destroyed.

In my opinion, it should be Deliverance's responsibility to make sure accessing the request object read-only does not destroy the request. It could be argued that this is easiest done in proxy_to_dest by restoring wsgi.input from request.POST if input was consumed.