http.server doesn't process Status header from CGI scripts

[b5a9ce10-d67f-478f-ab78-b08d099eb753]

BPO	10487
Nosy	@freddrake, @facundobatista, @orsenthil, @vadmium

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/orsenthil' closed_at = None created_at = labels = ['type-feature', 'library'] title = "http.server doesn't process Status header from CGI scripts" updated_at = user = 'https://bugs.python.org/vpython' ``` bugs.python.org fields: ```python activity = actor = 'martin.panter' assignee = 'orsenthil' closed = False closed_date = None closer = None components = ['Library (Lib)'] creation = creator = 'v+python' dependencies = [] files = [] hgrepos = [] issue_num = 10487 keywords = [] message_count = 3.0 messages = ['121881', '123515', '252071'] nosy_count = 6.0 nosy_names = ['fdrake', 'facundobatista', 'orsenthil', 'v+python', 'rosslagerwall', 'martin.panter'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue10487' versions = ['Python 3.3'] ```

[b5a9ce10-d67f-478f-ab78-b08d099eb753]

While it is documented that http.server (and Python 2's CGIHTTPServer) do not process the status header, and limit the usefulness of CGI scripts as a result, that doesn't make it less of a bug, just a documented bug. But I guess that it might have to be called a feature request; I'll not argue if someone switches this to feature request, but I consider it a bug.

See related bpo-10482 for subprocess to provide better features for avoiding deadlock situations. There seems to be no general way using subprocess to avoid possible deadlock situations. However, since CGI doesn't really use stderr much, and only for logging, which the scripts can do themselves (the cgi.py module even provides for such), and because CGIs generally slurp stdin before creating stdout, it is possible to tweak sidestep use of subprocess.communicate, drop the stdout PIPE, and sequence the code to process stdin and then stdout, and not generally deadlock (some CGI scripts that don't above the stdin before stdout rule, might deadlock if called with POST and large inputs, but those are few).

By doing this, one can then add code to handle Status: headers, and avoid buffering large files on output (and on input). The tradeoff is losing the stderr log; when that is hooked up, some error cases can trigger deadlocks by writing to stderr -- hence the subprocess issue mentioned above.

[b5a9ce10-d67f-478f-ab78-b08d099eb753]

Just to mention, with the added code from bpo-10482, I was able to get a 3-stream functionality working great in http.server and also backported it to 2.6 CGIHTTPServer... and to properly process the Status: header on stdout.

Works very well in 2.6; bpo-8077 prevents form processing from working in 3.2a4, but otherwise it is working there also, and the experience in 2.6 indicates that once bpo-8077 is resolved, it should work in 3.2 also.

[vadmium]

bpo-13893 was closed as a duplicate of this issue. There are patches and modified code there that may be worth looking at though.