search

gabrielfalcao / HTTPretty

Intercept HTTP requests at the Python socket level. Fakes the whole socket module
https://httpretty.readthedocs.org
MIT License
2.11k stars 276 forks source link

Separate headers by CR LF #471

Open dallagi opened 1 year ago

dallagi commented 1 year ago

While using HTTPretty together with the last version of aiohttp, aiohttp failed to parse the response generated by HTTPretty due to the way headers are separated. Specifically, HTTPretty separates them via the LF (\n) control code while aiohttp expects CR LF (\r\n).

This behavior (on aiohttp's side) is also mentioned here https://github.com/aio-libs/aiohttp/issues/7494#issuecomment-1668546182 .

By searching a bit on the Internet (eg. here) it seems that http parsers are expected to be lenient wrt parsing newlines, so I'm not sure this is technically a fault on HTTPretty's side.

However I think it could be better to stay on the safe side and just use \r\n as separator for headers, which is the change I implemented with this PR.

Dreamsorcerer commented 1 year ago

By searching a bit on the Internet (eg. here) it seems that http parsers are expected to be lenient wrt parsing newlines, so I'm not sure this is technically a fault on HTTPretty's side.

It's literally not an HTTP message, which is clearly a mistake for a library with HTTP in its name. ;)

HTTP-message = start-line CRLF *( field-line CRLF ) CRLF [ message-body ] https://www.rfc-editor.org/rfc/rfc9112.html#section-2.1-1

Allowing LF is completely optional, and can lead to security issues, particularly on the server side:

Although the line terminator for the start-line and fields is the sequence CRLF, a recipient MAY recognize a single LF as a line terminator and ignore any preceding CR. https://www.rfc-editor.org/rfc/rfc9112.html#section-2.2-3