search

aio-libs / aiohttp

Asynchronous HTTP client/server framework for asyncio and Python
https://docs.aiohttp.org
Other
15.18k stars 2.02k forks source link

Cannot remove "Server" header in all cases for security hardening #6463

Open hofst opened 2 years ago

hofst commented 2 years ago

Describe the bug

Context: I believe the recommended way to remove the Server header is via on_response_prepare. This works well in most but not all cases. Since the Server header is a security liability, it is important to remove it in all cases. But independently, it would be important to have a reliable mechanism/hook/callback to modify all responses. Note: The callback is called correctly for other errors, e.g., routing errors.

Problem: One noteworthy case where the callback does not work is the processing of BaseRequests which does not call the on_response_prepare callback. Unfortuantely, aiohttp internally uses BaseRequests in case of parser errors. You can provoke parser errors, e.g, via curl -X ASD localhost. This is not a contrived example. I have noticed such requests from some unknown attacker/bot in my logs today.

Independent but related issue: I tried to work around it by manually overwriting the BaseRequest._prepare_hook from outside but noticed another problem: It appears that an asyncio.CancelledError is not handled correctly if it is raised during a parsing errors from the _prepare_hook: the socket stays open and leaks (from the client's perspective).

To Reproduce

  1. Write a callback and register it in on_response_prepare
  2. Provoke a parsing error, e.g., via curl -X ASD localhost
  3. Notice that the on_resposne_prepare callback is not called

Expected behavior

on_resposne_prepare callbacks should always be called and for any Response

Logs/tracebacks

This is an exception example for a parsing error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/aiohttp/web_protocol.py", line 314, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp/_http_parser.pyx", line 546, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'invalid HTTP method'"

Python Version

3.9.5

aiohttp Version

Version: 3.7.4.post0

multidict Version

Version: 5.1.0

yarl Version

Version: 1.6.3

OS

Ubuntu 21.04

Related component

Server

Additional context

No response

Code of Conduct

Dreamsorcerer commented 3 months ago

At a glance, it looks like there is no match_info available in BaseRequest. I think the best solution would be to change middlewares to run even on system routes: https://github.com/aio-libs/aiohttp/issues/3287#issuecomment-2276937564