qnbhd / mljet

Minimalistic ML-models auto deployment tool
MIT License
67 stars 4 forks source link

chore(deps): update requirements to v3.10.2 [security] #149

Open renovate[bot] opened 1 year ago

renovate[bot] commented 1 year ago

This PR contains the following updates:

Package Change Age Adoption Passing Confidence
aiohttp ==3.8.4 -> ==3.10.2 age adoption passing confidence

GitHub Vulnerability Alerts

CVE-2023-37276

Impact

aiohttp v3.8.4 and earlier are bundled with llhttp v6.0.6 which is vulnerable to CVE-2023-30589. The vulnerable code is used by aiohttp for its HTTP request parser when available which is the default case when installing from a wheel.

This vulnerability only affects users of aiohttp as an HTTP server (ie aiohttp.Application), you are not affected by this vulnerability if you are using aiohttp as an HTTP client library (ie aiohttp.ClientSession).

Reproducer

from aiohttp import web

async def example(request: web.Request):
    headers = dict(request.headers)
    body = await request.content.read()
    return web.Response(text=f"headers: {headers} body: {body}")

app = web.Application()
app.add_routes([web.post('/', example)])
web.run_app(app)

Sending a crafted HTTP request will cause the server to misinterpret one of the HTTP header values leading to HTTP request smuggling.

$ printf "POST / HTTP/1.1\r\nHost: localhost:8080\r\nX-Abc: \rxTransfer-Encoding: chunked\r\n\r\n1\r\nA\r\n0\r\n\r\n" \
  | nc localhost 8080

Expected output:
  headers: {'Host': 'localhost:8080', 'X-Abc': '\rxTransfer-Encoding: chunked'} body: b''

Actual output (note that 'Transfer-Encoding: chunked' is an HTTP header now and body is treated differently)
  headers: {'Host': 'localhost:8080', 'X-Abc': '', 'Transfer-Encoding': 'chunked'} body: b'A'

Patches

Upgrade to the latest version of aiohttp to resolve this vulnerability. It has been fixed in v3.8.5: pip install aiohttp >= 3.8.5

Workarounds

If you aren't able to upgrade you can reinstall aiohttp using AIOHTTP_NO_EXTENSIONS=1 as an environment variable to disable the llhttp HTTP request parser implementation. The pure Python implementation isn't vulnerable to request smuggling:

$ python -m pip uninstall --yes aiohttp
$ AIOHTTP_NO_EXTENSIONS=1 python -m pip install --no-binary=aiohttp --no-cache aiohttp

References

CVE-2023-47627

Summary

The HTTP parser in AIOHTTP has numerous problems with header parsing, which could lead to request smuggling. This parser is only used when AIOHTTP_NO_EXTENSIONS is enabled (or not using a prebuilt wheel).

Details

Bug 1: Bad parsing of Content-Length values

Description

RFC 9110 says this:

Content-Length = 1*DIGIT

AIOHTTP does not enforce this rule, presumably because of an incorrect usage of the builtin int constructor. Because the int constructor accepts + and - prefixes, and digit-separating underscores, using int to parse CL values leads AIOHTTP to significant misinterpretation.

Examples

GET / HTTP/1.1\r\n
Content-Length: -0\r\n
\r\n
X
GET / HTTP/1.1\r\n
Content-Length: +0_1\r\n
\r\n
X

Suggested action

Verify that a Content-Length value consists only of ASCII digits before parsing, as the standard requires.

Bug 2: Improper handling of NUL, CR, and LF in header values

Description

RFC 9110 says this:

Field values containing CR, LF, or NUL characters are invalid and dangerous, due to the varying ways that implementations might parse and interpret those characters; a recipient of CR, LF, or NUL within a field value MUST either reject the message or replace each of those characters with SP before further processing or forwarding of that message.

AIOHTTP's HTTP parser does not enforce this rule, and will happily process header values containing these three forbidden characters without replacing them with SP.

Examples

GET / HTTP/1.1\r\n
Header: v\x00alue\r\n
\r\n
GET / HTTP/1.1\r\n
Header: v\ralue\r\n
\r\n
GET / HTTP/1.1\r\n
Header: v\nalue\r\n
\r\n

Suggested action

Reject all messages with NUL, CR, or LF in a header value. The translation to space thing, while technically allowed, does not seem like a good idea to me.

Bug 3: Improper stripping of whitespace before colon in HTTP headers

Description

RFC 9112 says this:

No whitespace is allowed between the field name and colon. In the past, differences in the handling of such whitespace have led to security vulnerabilities in request routing and response handling. A server MUST reject, with a response status code of 400 (Bad Request), any received request message that contains whitespace between a header field name and colon.

AIOHTTP does not enforce this rule, and will simply strip any whitespace before the colon in an HTTP header.

Example

GET / HTTP/1.1\r\n
Content-Length : 1\r\n
\r\n
X

Suggested action

Reject all messages with whitespace before a colon in a header field, as the standard requires.

PoC

Example requests are embedded in the previous section. To reproduce these bugs, start an AIOHTTP server without llhttp (i.e. AIOHTTP_NO_EXTENSIONS=1) and send the requests given in the previous section. (e.g. by printfing into nc)

Impact

Each of these bugs can be used for request smuggling.

GHSA-pjjw-qhg8-p2p9

Summary

llhttp 8.1.1 is vulnerable to two request smuggling vulnerabilities. Details have not been disclosed yet, so refer to llhttp for future information. The issue is resolved by using llhttp 9+ (which is included in aiohttp 3.8.6+).

CVE-2023-49082

Summary

Improper validation makes it possible for an attacker to modify the HTTP request (e.g. insert a new header) or even create a new HTTP request if the attacker controls the HTTP method.

Details

The vulnerability occurs only if the attacker can control the HTTP method (GET, POST etc.) of the request.

Previous releases performed no validation on the provided value. If an attacker controls the HTTP method it will be used as is and can lead to HTTP request smuggling.

PoC

A minimal example can be found here: https://gist.github.com/jnovikov/7f411ae9fe6a9a7804cf162a3bdbb44b

Impact

If the attacker can control the HTTP version of the request it will be able to modify the request (request smuggling).

Workaround

If unable to upgrade and using user-provided values for the request method, perform manual validation of the user value (e.g. by restricting it to a few known values like GET, POST etc.).

Patch: https://github.com/aio-libs/aiohttp/pull/7806/files

CVE-2023-49081

Summary

Improper validation make it possible for an attacker to modify the HTTP request (e.g. to insert a new header) or even create a new HTTP request if the attacker controls the HTTP version.

Details

The vulnerability only occurs if the attacker can control the HTTP version of the request (including its type). For example if an unvalidated JSON value is used as a version and the attacker is then able to pass an array as the version parameter. Furthermore, the vulnerability only occurs when the Connection header is passed to the headers parameter.

At this point, the library will use the parsed value to create the request. If a list is passed, then it bypasses validation and it is possible to perform CRLF injection.

PoC

The POC below shows an example of providing an unvalidated array as a version: https://gist.github.com/jnovikov/184afb593d9c2114d77f508e0ccd508e

Impact

CRLF injection leading to Request Smuggling.

Workaround

If these specific conditions are met and you are unable to upgrade, then validate the user input to the version parameter to ensure it is a str.

Patch: https://github.com/aio-libs/aiohttp/pull/7835/files

CVE-2024-23829

Summary

Security-sensitive parts of the Python HTTP parser retained minor differences in allowable character sets, that must trigger error handling to robustly match frame boundaries of proxies in order to protect against injection of additional requests. Additionally, validation could trigger exceptions that were not handled consistently with processing of other malformed input.

Details

These problems are rooted in pattern matching protocol elements, previously improved by PR #​3235 and GHSA-gfw2-4jvh-wgfg:

  1. The expression HTTP/(\d).(\d) lacked another backslash to clarify that the separator should be a literal dot, not just any Unicode code point (result: HTTP/(\d)\.(\d)).

  2. The HTTP version was permitting Unicode digits, where only ASCII digits are standards-compliant.

  3. Distinct regular expressions for validating HTTP Method and Header field names were used - though both should (at least) apply the common restrictions of rfc9110 token.

PoC

GET / HTTP/1ö1 GET / HTTP/1.𝟙 GET/: HTTP/1.1 Content-Encoding?: chunked

Impact

Primarily concerns running an aiohttp server without llhttp:

  1. behind a proxy: Being more lenient than internet standards require could, depending on deployment environment, assist in request smuggling.
  2. directly accessible or exposed behind proxies relaying malformed input: the unhandled exception could cause excessive resource consumption on the application server and/or its logging facilities.

Patch: https://github.com/aio-libs/aiohttp/pull/8074/files

CVE-2024-23334

Summary

Improperly configuring static resource resolution in aiohttp when used as a web server can result in the unauthorized reading of arbitrary files on the system.

Details

When using aiohttp as a web server and configuring static routes, it is necessary to specify the root path for static files. Additionally, the option 'follow_symlinks' can be used to determine whether to follow symbolic links outside the static root directory. When 'follow_symlinks' is set to True, there is no validation to check if a given file path is within the root directory.This can lead to directory traversal vulnerabilities, resulting in unauthorized access to arbitrary files on the system, even when symlinks are not present.

i.e. An application is only vulnerable with setup code like:

app.router.add_routes([
    web.static("/static", "static/", follow_symlinks=True),  # Remove follow_symlinks to avoid the vulnerability
])

Impact

This is a directory traversal vulnerability with CWE ID 22. When using aiohttp as a web server and enabling static resource resolution with follow_symlinks set to True, it can lead to this vulnerability. This vulnerability has been present since the introduction of the follow_symlinks parameter.

Workaround

Even if upgrading to a patched version of aiohttp, we recommend following these steps regardless.

If using follow_symlinks=True outside of a restricted local development environment, disable the option immediately. This option is NOT needed to follow symlinks which point to a location within the static root directory, it is only intended to allow a symlink to break out of the static directory. Even with this CVE fixed, there is still a substantial risk of misconfiguration when using this option on a server that accepts requests from remote users.

Additionally, aiohttp has always recommended using a reverse proxy server (such as nginx) to handle static resources and not to use these static resources in aiohttp for production environments. Doing so also protects against this vulnerability, and is why we expect the number of affected users to be very low.


Patch: https://github.com/aio-libs/aiohttp/pull/8079/files

CVE-2024-27306

Summary

A XSS vulnerability exists on index pages for static file handling.

Details

When using web.static(..., show_index=True), the resulting index pages do not escape file names.

If users can upload files with arbitrary filenames to the static directory, the server is vulnerable to XSS attacks.

Workaround

We have always recommended using a reverse proxy server (e.g. nginx) for serving static files. Users following the recommendation are unaffected.

Other users can disable show_index if unable to upgrade.


Patch: https://github.com/aio-libs/aiohttp/pull/8319/files

CVE-2024-30251

Summary

An attacker can send a specially crafted POST (multipart/form-data) request. When the aiohttp server processes it, the server will enter an infinite loop and be unable to process any further requests.

Impact

An attacker can stop the application from serving requests after sending a single request.


For anyone needing to patch older versions of aiohttp, the minimum diff needed to resolve the issue is (located in _read_chunk_from_length()):

diff --git a/aiohttp/multipart.py b/aiohttp/multipart.py
index 227be605c..71fc2654a 100644
--- a/aiohttp/multipart.py
+++ b/aiohttp/multipart.py
@​@​ -338,6 +338,8 @​@​ class BodyPartReader:
         assert self._length is not None, "Content-Length required for chunked read"
         chunk_size = min(size, self._length - self._read_bytes)
         chunk = await self._content.read(chunk_size)
+        if self._content.at_eof():
+            self._at_eof = True
         return chunk

     async def _read_chunk_from_stream(self, size: int) -> bytes:

This does however introduce some very minor issues with handling form data. So, if possible, it would be recommended to also backport the changes in: https://github.com/aio-libs/aiohttp/commit/cebe526b9c34dc3a3da9140409db63014bc4cf19 https://github.com/aio-libs/aiohttp/commit/7eecdff163ccf029fbb1ddc9de4169d4aaeb6597 https://github.com/aio-libs/aiohttp/commit/f21c6f2ca512a026ce7f0f6c6311f62d6a638866

CVE-2024-42367

Summary

Static routes which contain files with compressed variants (.gz or .br extension) were vulnerable to path traversal outside the root directory if those variants are symbolic links.

Details

The server protects static routes from path traversal outside the root directory when follow_symlinks=False (default). It does this by resolving the requested URL to an absolute path and then checking that path relative to the root. However, these checks are not performed when looking for compressed variants in the FileResponse class, and symbolic links are then automatically followed when performing Path.stat() and Path.open() to send the file.

Impact

Servers with static routes that contain compressed variants as symbolic links, pointing outside the root directory, or that permit users to upload or create such links, are impacted.


Patch: https://github.com/aio-libs/aiohttp/pull/8653/files


Release Notes

aio-libs/aiohttp (aiohttp) ### [`v3.10.2`](https://redirect.github.com/aio-libs/aiohttp/blob/HEAD/CHANGES.rst#3102-2024-08-08) [Compare Source](https://redirect.github.com/aio-libs/aiohttp/compare/v3.10.1...v3.10.2) \=================== ## Bug fixes - Fixed server checks for circular symbolic links to be compatible with Python 3.13 -- by :user:`steverep`. *Related issues and pull requests on GitHub:* :issue:`8565`. - Fixed request body not being read when ignoring an Upgrade request -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8597`. - Fixed an edge case where shutdown would wait for timeout when the handler was already completed -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8611`. - Fixed connecting to `npipe://`, `tcp://`, and `unix://` urls -- by :user:`bdraco`. *Related issues and pull requests on GitHub:* :issue:`8632`. - Fixed WebSocket ping tasks being prematurely garbage collected -- by :user:`bdraco`. There was a small risk that WebSocket ping tasks would be prematurely garbage collected because the event loop only holds a weak reference to the task. The garbage collection risk has been fixed by holding a strong reference to the task. Additionally, the task is now scheduled eagerly with Python 3.12+ to increase the chance it can be completed immediately and avoid having to hold any references to the task. *Related issues and pull requests on GitHub:* :issue:`8641`. - Fixed incorrectly following symlinks for compressed file variants -- by :user:`steverep`. *Related issues and pull requests on GitHub:* :issue:`8652`. ## Removals and backward incompatible breaking changes - Removed `Request.wait_for_disconnection()`, which was mistakenly added briefly in 3.10.0 -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8636`. ## Contributor-facing changes - Fixed monkey patches for `Path.stat()` and `Path.is_dir()` for Python 3.13 compatibility -- by :user:`steverep`. *Related issues and pull requests on GitHub:* :issue:`8551`. ## Miscellaneous internal changes - Improved WebSocket performance when messages are sent or received frequently -- by :user:`bdraco`. The WebSocket heartbeat scheduling algorithm was improved to reduce the `asyncio` scheduling overhead by decreasing the number of `asyncio.TimerHandle` creations and cancellations. *Related issues and pull requests on GitHub:* :issue:`8608`. - Minor improvements to various type annotations -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8634`. *** ### [`v3.10.1`](https://redirect.github.com/aio-libs/aiohttp/compare/v3.10.0...v3.10.1) [Compare Source](https://redirect.github.com/aio-libs/aiohttp/compare/v3.10.0...v3.10.1) ### [`v3.10.0`](https://redirect.github.com/aio-libs/aiohttp/compare/v3.9.5...v3.10.0) [Compare Source](https://redirect.github.com/aio-libs/aiohttp/compare/v3.9.5...v3.10.0) ### [`v3.9.5`](https://redirect.github.com/aio-libs/aiohttp/blob/HEAD/CHANGES.rst#395-2024-04-16) [Compare Source](https://redirect.github.com/aio-libs/aiohttp/compare/v3.9.4...v3.9.5) \================== ## Bug fixes - Fixed "Unclosed client session" when initialization of :py:class:`~aiohttp.ClientSession` fails -- by :user:`NewGlad`. *Related issues and pull requests on GitHub:* :issue:`8253`. - Fixed regression (from :pr:`8280`) with adding `Content-Disposition` to the `form-data` part after appending to writer -- by :user:`Dreamsorcerer`/:user:`Olegt0rr`. *Related issues and pull requests on GitHub:* :issue:`8332`. - Added default `Content-Disposition` in `multipart/form-data` responses to avoid broken form-data responses -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8335`. *** ### [`v3.9.4`](https://redirect.github.com/aio-libs/aiohttp/blob/HEAD/CHANGES.rst#394-2024-04-11) [Compare Source](https://redirect.github.com/aio-libs/aiohttp/compare/v3.9.3...v3.9.4) \================== ## Bug fixes - The asynchronous internals now set the underlying causes when assigning exceptions to the future objects \-- by :user:`webknjaz`. *Related issues and pull requests on GitHub:* :issue:`8089`. - Treated values of `Accept-Encoding` header as case-insensitive when checking for gzip files -- by :user:`steverep`. *Related issues and pull requests on GitHub:* :issue:`8104`. - Improved the DNS resolution performance on cache hit -- by :user:`bdraco`. This is achieved by avoiding an :mod:`asyncio` task creation in this case. *Related issues and pull requests on GitHub:* :issue:`8163`. - Changed the type annotations to allow `dict` on :meth:`aiohttp.MultipartWriter.append`, :meth:`aiohttp.MultipartWriter.append_json` and :meth:`aiohttp.MultipartWriter.append_form` -- by :user:`cakemanny` *Related issues and pull requests on GitHub:* :issue:`7741`. - Ensure websocket transport is closed when client does not close it \-- by :user:`bdraco`. The transport could remain open if the client did not close it. This change ensures the transport is closed when the client does not close it. *Related issues and pull requests on GitHub:* :issue:`8200`. - Leave websocket transport open if receive times out or is cancelled \-- by :user:`bdraco`. This restores the behavior prior to the change in [#​7978](https://redirect.github.com/aio-libs/aiohttp/issues/7978). *Related issues and pull requests on GitHub:* :issue:`8251`. - Fixed content not being read when an upgrade request was not supported with the pure Python implementation. \-- by :user:`bdraco`. *Related issues and pull requests on GitHub:* :issue:`8252`. - Fixed a race condition with incoming connections during server shutdown -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8271`. - Fixed `multipart/form-data` compliance with :rfc:`7578` -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8280`. - Fixed blocking I/O in the event loop while processing files in a POST request \-- by :user:`bdraco`. *Related issues and pull requests on GitHub:* :issue:`8283`. - Escaped filenames in static view -- by :user:`bdraco`. *Related issues and pull requests on GitHub:* :issue:`8317`. - Fixed the pure python parser to mark a connection as closing when a response has no length -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8320`. ## Features - Upgraded *llhttp* to 9.2.1, and started rejecting obsolete line folding in Python parser to match -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8146`, :issue:`8292`. ## Deprecations (removal in next major release) - Deprecated `content_transfer_encoding` parameter in :py:meth:`FormData.add_field() ` -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8280`. ## Improved documentation - Added a note about canceling tasks to avoid delaying server shutdown -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8267`. ## Contributor-facing changes - The pull request template is now asking the contributors to answer a question about the long-term maintenance challenges they envision as a result of merging their patches \-- by :user:`webknjaz`. *Related issues and pull requests on GitHub:* :issue:`8099`. - Updated CI and documentation to use NPM clean install and upgrade node to version 18 -- by :user:`steverep`. *Related issues and pull requests on GitHub:* :issue:`8116`. - A pytest fixture `hello_txt` was introduced to aid static file serving tests in :file:`test_web_sendfile_functional.py`. It dynamically provisions `hello.txt` file variants shared across the tests in the module. \-- by :user:`steverep` *Related issues and pull requests on GitHub:* :issue:`8136`. ## Packaging updates and notes for downstreams - Added an `internal` pytest marker for tests which should be skipped by packagers (use `-m 'not internal'` to disable them) -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8299`. *** ### [`v3.9.3`](https://redirect.github.com/aio-libs/aiohttp/blob/HEAD/CHANGES.rst#393-2024-01-29) [Compare Source](https://redirect.github.com/aio-libs/aiohttp/compare/v3.9.2...v3.9.3) \================== ## Bug fixes - Fixed backwards compatibility breakage (in 3.9.2) of `ssl` parameter when set outside of `ClientSession` (e.g. directly in `TCPConnector`) -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`8097`, :issue:`8098`. ## Miscellaneous internal changes - Improved test suite handling of paths and temp files to consistently use pathlib and pytest fixtures. *Related issues and pull requests on GitHub:* :issue:`3957`. *** ### [`v3.9.2`](https://redirect.github.com/aio-libs/aiohttp/blob/HEAD/CHANGES.rst#392-2024-01-28) [Compare Source](https://redirect.github.com/aio-libs/aiohttp/compare/v3.9.1...v3.9.2) \================== ## Bug fixes - Fixed server-side websocket connection leak. *Related issues and pull requests on GitHub:* :issue:`7978`. - Fixed `web.FileResponse` doing blocking I/O in the event loop. *Related issues and pull requests on GitHub:* :issue:`8012`. - Fixed double compress when compression enabled and compressed file exists in server file responses. *Related issues and pull requests on GitHub:* :issue:`8014`. - Added runtime type check for `ClientSession` `timeout` parameter. *Related issues and pull requests on GitHub:* :issue:`8021`. - Fixed an unhandled exception in the Python HTTP parser on header lines starting with a colon -- by :user:`pajod`. Invalid request lines with anything but a dot between the HTTP major and minor version are now rejected. Invalid header field names containing question mark or slash are now rejected. Such requests are incompatible with :rfc:`9110#section-5.6.2` and are not known to be of any legitimate use. *Related issues and pull requests on GitHub:* :issue:`8074`. - Improved validation of paths for static resources requests to the server -- by :user:`bdraco`. *Related issues and pull requests on GitHub:* :issue:`8079`. ## Features - Added support for passing :py:data:`True` to `ssl` parameter in `ClientSession` while deprecating :py:data:`None` -- by :user:`xiangyan99`. *Related issues and pull requests on GitHub:* :issue:`7698`. ## Breaking changes - Fixed an unhandled exception in the Python HTTP parser on header lines starting with a colon -- by :user:`pajod`. Invalid request lines with anything but a dot between the HTTP major and minor version are now rejected. Invalid header field names containing question mark or slash are now rejected. Such requests are incompatible with :rfc:`9110#section-5.6.2` and are not known to be of any legitimate use. *Related issues and pull requests on GitHub:* :issue:`8074`. ## Improved documentation - Fixed examples of `fallback_charset_resolver` function in the :doc:`client_advanced` document. -- by :user:`henry0312`. *Related issues and pull requests on GitHub:* :issue:`7995`. - The Sphinx setup was updated to avoid showing the empty changelog draft section in the tagged release documentation builds on Read The Docs -- by :user:`webknjaz`. *Related issues and pull requests on GitHub:* :issue:`8067`. ## Packaging updates and notes for downstreams - The changelog categorization was made clearer. The contributors can now mark their fragment files more accurately -- by :user:`webknjaz`. The new category tags are: * ``bugfix`` * ``feature`` * ``deprecation`` * ``breaking`` (previously, ``removal``) * ``doc`` * ``packaging`` * ``contrib`` * ``misc`` *Related issues and pull requests on GitHub:* :issue:`8066`. ## Contributor-facing changes - Updated :ref:`contributing/Tests coverage ` section to show how we use `codecov` -- by :user:`Dreamsorcerer`. *Related issues and pull requests on GitHub:* :issue:`7916`. - The changelog categorization was made clearer. The contributors can now mark their fragment files more accurately -- by :user:`webknjaz`. The new category tags are: * ``bugfix`` * ``feature`` * ``deprecation`` * ``breaking`` (previously, ``removal``) * ``doc`` * ``packaging`` * ``contrib`` * ``misc`` *Related issues and pull requests on GitHub:* :issue:`8066`. ## Miscellaneous internal changes - Replaced all `tmpdir` fixtures with `tmp_path` in test suite. *Related issues and pull requests on GitHub:* :issue:`3551`. *** ### [`v3.9.1`](https://redirect.github.com/aio-libs/aiohttp/blob/HEAD/CHANGES.rst#391-2023-11-26) [Compare Source](https://redirect.github.com/aio-libs/aiohttp/compare/v3.9.0...v3.9.1) \================== ## Bugfixes - Fixed importing aiohttp under PyPy on Windows. `#​7848 `\_ - Fixed async concurrency safety in websocket compressor. `#​7865 `\_ - Fixed `ClientResponse.close()` releasing the connection instead of closing. `#​7869 `\_ - Fixed a regression where connection may get closed during upgrade. -- by :user:`Dreamsorcerer` `#​7879 `\_ - Fixed messages being reported as upgraded without an Upgrade header in Python parser. -- by :user:`Dreamsorcerer` `#​7895 `\_ *** ### [`v3.9.0`](https://redirect.github.com/aio-libs/aiohttp/blob/HEAD/CHANGES.rst#390-2023-11-18) [Compare Source](https://redirect.github.com/aio-libs/aiohttp/compare/v3.8.6...v3.9.0) \================== ## Features - Introduced `AppKey` for static typing support of `Application` storage. See https://docs.aiohttp.org/en/stable/web_advanced.html#application-s-config `#​5864 `\_ - Added a graceful shutdown period which allows pending tasks to complete before the application's cleanup is called. The period can be adjusted with the `shutdown_timeout` parameter. -- by :user:`Dreamsorcerer`. See https://docs.aiohttp.org/en/latest/web_advanced.html#graceful-shutdown `#​7188 `\_ - Added `handler_cancellation `\_ parameter to cancel web handler on client disconnection. -- by :user:`mosquito` This (optionally) reintroduces a feature removed in a previous release. Recommended for those looking for an extra level of protection against denial-of-service attacks. `#​7056 `\_ - Added support for setting response header parameters `max_line_size` and `max_field_size`. `#​2304 `\_ - Added `auto_decompress` parameter to `ClientSession.request` to override `ClientSession._auto_decompress`. -- by :user:`Daste745` `#​3751 `\_ - Changed `raise_for_status` to allow a coroutine. `#​3892 `\_ - Added client brotli compression support (optional with runtime check). `#​5219 `\_ - Added `client_max_size` to `BaseRequest.clone()` to allow overriding the request body size. -- :user:`anesabml`. `#​5704 `\_ - Added a middleware type alias `aiohttp.typedefs.Middleware`. `#​5898 `\_ - Exported `HTTPMove` which can be used to catch any redirection request that has a location -- :user:`dreamsorcerer`. `#​6594 `\_ - Changed the `path` parameter in `web.run_app()` to accept a `pathlib.Path` object. `#​6839 `\_ - Performance: Skipped filtering `CookieJar` when the jar is empty or all cookies have expired. `#​7819 `\_ - Performance: Only check origin if insecure scheme and there are origins to treat as secure, in `CookieJar.filter_cookies()`. `#​7821 `\_ - Performance: Used timestamp instead of `datetime` to achieve faster cookie expiration in `CookieJar`. `#​7824 `\_ - Added support for passing a custom server name parameter to HTTPS connection. `#​7114 `\_ - Added support for using Basic Auth credentials from :file:`.netrc` file when making HTTP requests with the :py:class:`~aiohttp.ClientSession` `trust_env` argument is set to `True`. -- by :user:`yuvipanda`. `#​7131 `\_ - Turned access log into no-op when the logger is disabled. `#​7240 `\_ - Added typing information to `RawResponseMessage`. -- by :user:`Gobot1234` `#​7365 `\_ - Removed `async-timeout` for Python 3.11+ (replaced with `asyncio.timeout()` on newer releases). `#​7502 `\_ - Added support for `brotlicffi` as an alternative to `brotli` (fixing Brotli support on PyPy). `#​7611 `\_ - Added `WebSocketResponse.get_extra_info()` to access a protocol transport's extra info. `#​7078 `\_ - Allow `link` argument to be set to None/empty in HTTP 451 exception. `#​7689 `\_ ## Bugfixes - Implemented stripping the trailing dots from fully-qualified domain names in `Host` headers and TLS context when acting as an HTTP client. This allows the client to connect to URLs with FQDN host name like `https://example.com./`. \-- by :user:`martin-sucha`. `#​3636 `\_ - Fixed client timeout not working when incoming data is always available without waiting. -- by :user:`Dreamsorcerer`. `#​5854 `\_ - Fixed `readuntil` to work with a delimiter of more than one character. `#​6701 `\_ - Added `__repr__` to `EmptyStreamReader` to avoid `AttributeError`. `#​6916 `\_ - Fixed bug when using `TCPConnector` with `ttl_dns_cache=0`. `#​7014 `\_ - Fixed response returned from expect handler being thrown away. -- by :user:`Dreamsorcerer` `#​7025 `\_ - Avoided raising `UnicodeDecodeError` in multipart and in HTTP headers parsing. `#​7044 `\_ - Changed `sock_read` timeout to start after writing has finished, avoiding read timeouts caused by an unfinished write. -- by :user:`dtrifiro` `#​7149 `\_ - Fixed missing query in tracing method URLs when using `yarl` 1.9+. `#​7259 `\_ - Changed max 32-bit timestamp to an aware datetime object, for consistency with the non-32-bit one, and to avoid a `DeprecationWarning` on Python 3.12. `#​7302 `\_ - Fixed `EmptyStreamReader.iter_chunks()` never ending. -- by :user:`mind1m` `#​7616 `\_ - Fixed a rare `RuntimeError: await wasn't used with future` exception. -- by :user:`stalkerg` `#​7785 `\_ - Fixed issue with insufficient HTTP method and version validation. `#​7700 `\_ - Added check to validate that absolute URIs have schemes. `#​7712 `\_ - Fixed unhandled exception when Python HTTP parser encounters unpaired Unicode surrogates. `#​7715 `\_ - Updated parser to disallow invalid characters in header field names and stop accepting LF as a request line separator. `#​7719 `\_ - Fixed Python HTTP parser not treating 204/304/1xx as an empty body. `#​7755 `\_ - Ensure empty body response for 1xx/204/304 per RFC 9112 sec 6.3. `#​7756 `\_ - Fixed an issue when a client request is closed before completing a chunked payload. -- by :user:`Dreamsorcerer` `#​7764 `\_ - Edge Case Handling for ResponseParser for missing reason value. `#​7776 `\_ - Fixed `ClientWebSocketResponse.close_code` being erroneously set to `None` when there are concurrent async tasks receiving data and closing the connection. `#​7306 `\_ - Added HTTP method validation. `#​6533 `\_ - Fixed arbitrary sequence types being allowed to inject values via version parameter. -- by :user:`Dreamsorcerer` `#​7835 `\_ - Performance: Fixed increase in latency with small messages from websocket compression changes. `#​7797 `\_ ## Improved Documentation - Fixed the `ClientResponse.release`'s type in the doc. Changed from `comethod` to `method`. `#​5836 `\_ - Added information on behavior of base_url parameter in `ClientSession`. `#​6647 `\_ - Fixed `ClientResponseError` docs. `#​6700 `\_ - Updated Redis code examples to follow the latest API. `#​6907 `\_ - Added a note about possibly needing to update headers when using `on_response_prepare`. -- by :user:`Dreamsorcerer` `#​7283 `\_ - Completed `trust_env` parameter description to honor `wss_proxy`, `ws_proxy` or `no_proxy` env. `#​7325 `\_ - Expanded SSL documentation with more examples (e.g. how to use certifi). -- by :user:`Dreamsorcerer` `#​7334 `\_ - Fix, update, and improve client exceptions documentation. `#​7733 `\_ ## Deprecations and Removals - Added `shutdown_timeout` parameter to `BaseRunner`, while deprecating `shutdown_timeout` parameter from `BaseSite`. -- by :user:`Dreamsorcerer` `#​7718 `\_ - Dropped Python 3.6 support. `#​6378 `\_ - Dropped Python 3.7 support. -- by :user:`Dreamsorcerer` `#​7336 `\_ - Removed support for abandoned `tokio` event loop. -- by :user:`Dreamsorcerer` `#​7281 `\_ ## Misc - Made `print` argument in `run_app()` optional. `#​3690 `\_ - Improved performance of `ceil_timeout` in some cases. `#​6316 `\_ - Changed importing Gunicorn to happen on-demand, decreasing import time by ~53%. -- :user:`Dreamsorcerer` `#​6591 `\_ - Improved import time by replacing `http.server` with `http.HTTPStatus`. `#​6903 `\_ - Fixed annotation of `ssl` parameter to disallow `True`. -- by :user:`Dreamsorcerer`. `#​7335 `\_ *** ### [`v3.8.6`](https://redirect.github.com/aio-libs/aiohttp/blob/HEAD/CHANGES.rst#386-2023-10-07) [Compare Source](https://redirect.github.com/aio-libs/aiohttp/compare/v3.8.5...v3.8.6) \================== ## Security bugfixes - Upgraded the vendored copy of llhttp\_ to v9.1.3 -- by :user:`Dreamsorcerer` Thanks to :user:`kenballus` for reporting this, see https://github.com/aio-libs/aiohttp/security/advisories/GHSA-pjjw-qhg8-p2p9. .. \_llhttp: https://llhttp.org `#​7647 `\_ - Updated Python parser to comply with RFCs 9110/9112 -- by :user:`Dreamorcerer` Thanks to :user:`kenballus` for reporting this, see https://github.com/aio-libs/aiohttp/security/advisories/GHSA-gfw2-4jvh-wgfg. `#​7663 `\_ ## Deprecation - Added `fallback_charset_resolver` parameter in `ClientSession` to allow a user-supplied character set detection function. Character set detection will no longer be included in 3.9 as a default. If this feature is needed, please use `fallback_charset_resolver `\_. `#​7561 `\_ ## Features - Enabled lenient response parsing for more flexible parsing in the client (this should resolve some regressions when dealing with badly formatted HTTP responses). -- by :user:`Dreamsorcerer` `#​7490 `\_ ## Bugfixes - Fixed `PermissionError` when `.netrc` is unreadable due to permissions. `#​7237 `\_ - Fixed output of parsing errors pointing to a `\n`. -- by :user:`Dreamsorcerer` `#​7468 `\_ - Fixed `GunicornWebWorker` max_requests_jitter not working. `#​7518 `\_ - Fixed sorting in `filter_cookies` to use cookie with longest path. -- by :user:`marq24`. `#​7577 `\_ - Fixed display of `BadStatusLine` messages from llhttp\_. -- by :user:`Dreamsorcerer` `#​7651 `\_ *** ### [`v3.8.5`](https://redirect.github.com/aio-libs/aiohttp/releases/tag/v3.8.5): 3.8.5 [Compare Source](https://redirect.github.com/aio-libs/aiohttp/compare/v3.8.4...v3.8.5) ## Security bugfixes - Upgraded the vendored copy of llhttp\_ to v8.1.1 -- by :user:`webknjaz` and :user:`Dreamsorcerer`. Thanks to :user:`sethmlarson` for reporting this and providing us with comprehensive reproducer, workarounds and fixing details! For more information, see https://github.com/aio-libs/aiohttp/security/advisories/GHSA-45c4-8wx5-qw6w. .. \_llhttp: https://llhttp.org ([#​7346](https://redirect.github.com/aio-libs/aiohttp/issues/7346)) ## Features - Added information to C parser exceptions to show which character caused the error. -- by :user:`Dreamsorcerer` ([#​7366](https://redirect.github.com/aio-libs/aiohttp/issues/7366)) ## Bugfixes - Fixed a transport is :data:`None` error -- by :user:`Dreamsorcerer`. ([#​3355](https://redirect.github.com/aio-libs/aiohttp/issues/3355)) ***

Configuration

📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.



This PR was generated by Mend Renovate. View the repository job log.