Since Requests v2.3.0, Requests has been vulnerable to potentially leaking Proxy-Authorization headers to destination servers, specifically during redirects to an HTTPS origin. This is a product of how rebuild_proxies is used to recompute and reattach the Proxy-Authorization header to requests when redirected. Note this behavior has only been observed to affect proxied requests when credentials are supplied in the URL user information component (e.g. https://username:password@proxy:8080).
Current vulnerable behavior(s):
HTTP → HTTPS: leak
HTTPS → HTTP: no leak
HTTPS → HTTPS: leak
HTTP → HTTP: no leak
For HTTP connections sent through the proxy, the proxy will identify the header in the request itself and remove it prior to forwarding to the destination server. However when sent over HTTPS, the Proxy-Authorization header must be sent in the CONNECT request as the proxy has no visibility into further tunneled requests. This results in Requests forwarding the header to the destination server unintentionally, allowing a malicious actor to potentially exfiltrate those credentials.
The reason this currently works for HTTPS connections in Requests is the Proxy-Authorization header is also handled by urllib3 with our usage of the ProxyManager in adapters.py with proxy_manager_for. This will compute the required proxy headers in proxy_headers and pass them to the Proxy Manager, avoiding attaching them directly to the Request object. This will be our preferred option going forward for default usage.
Patches
Starting in Requests v2.31.0, Requests will no longer attach this header to redirects with an HTTPS destination. This should have no negative impacts on the default behavior of the library as the proxy credentials are already properly being handled by urllib3's ProxyManager.
For users with custom adapters, this may be potentially breaking if you were already working around this behavior. The previous functionality of rebuild_proxies doesn't make sense in any case, so we would encourage any users impacted to migrate any handling of Proxy-Authorization directly into their custom adapter.
Workarounds
For users who are not able to update Requests immediately, there is one potential workaround.
You may disable redirects by setting allow_redirects to False on all calls through Requests top-level APIs. Note that if you're currently relying on redirect behaviors, you will need to capture the 3xx response codes and ensure a new request is made to the redirect destination.
import requests
r = requests.get('http://github.com/', allow_redirects=False)
Credits
This vulnerability was discovered and disclosed by the following individuals.
Dennis Brinkrolf, Haxolot (https://haxolot.com/)
Tobias Funke, (tobiasfunke93@gmail.com)
Release Notes
psf/requests (requests)
### [`v2.31.0`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2310-2023-05-22)
[Compare Source](https://togithub.com/psf/requests/compare/v2.30.0...v2.31.0)
**Security**
- Versions of Requests between v2.3.0 and v2.30.0 are vulnerable to potential
forwarding of `Proxy-Authorization` headers to destination servers when
following HTTPS redirects.
When proxies are defined with user info (https://user:pass@proxy:8080), Requests
will construct a `Proxy-Authorization` header that is attached to the request to
authenticate with the proxy.
In cases where Requests receives a redirect response, it previously reattached
the `Proxy-Authorization` header incorrectly, resulting in the value being
sent through the tunneled connection to the destination server. Users who rely on
defining their proxy credentials in the URL are *strongly* encouraged to upgrade
to Requests 2.31.0+ to prevent unintentional leakage and rotate their proxy
credentials once the change has been fully deployed.
Users who do not use a proxy or do not supply their proxy credentials through
the user information portion of their proxy URL are not subject to this
vulnerability.
Full details can be read in our [Github Security Advisory](https://togithub.com/psf/requests/security/advisories/GHSA-j8r2-6x86-q33q)
and [CVE-2023-32681](https://nvd.nist.gov/vuln/detail/CVE-2023-32681).
### [`v2.30.0`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2300-2023-05-03)
[Compare Source](https://togithub.com/psf/requests/compare/v2.29.0...v2.30.0)
**Dependencies**
- ⚠️ Added support for urllib3 2.0. ⚠️
This may contain minor breaking changes so we advise careful testing and
reviewing https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html
prior to upgrading.
Users who wish to stay on urllib3 1.x can pin to `urllib3<2`.
### [`v2.29.0`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2290-2023-04-26)
[Compare Source](https://togithub.com/psf/requests/compare/v2.28.2...v2.29.0)
**Improvements**
- Requests now defers chunked requests to the urllib3 implementation to improve
standardization. ([#6226](https://togithub.com/psf/requests/issues/6226))
- Requests relaxes header component requirements to support bytes/str subclasses. ([#6356](https://togithub.com/psf/requests/issues/6356))
### [`v2.28.2`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2282-2023-01-12)
[Compare Source](https://togithub.com/psf/requests/compare/v2.28.1...v2.28.2)
**Dependencies**
- Requests now supports charset_normalizer 3.x. ([#6261](https://togithub.com/psf/requests/issues/6261))
**Bugfixes**
- Updated MissingSchema exception to suggest https scheme rather than http. ([#6188](https://togithub.com/psf/requests/issues/6188))
### [`v2.28.1`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2281-2022-06-29)
[Compare Source](https://togithub.com/psf/requests/compare/v2.28.0...v2.28.1)
**Improvements**
- Speed optimization in `iter_content` with transition to `yield from`. ([#6170](https://togithub.com/psf/requests/issues/6170))
**Dependencies**
- Added support for chardet 5.0.0 ([#6179](https://togithub.com/psf/requests/issues/6179))
- Added support for charset-normalizer 2.1.0 ([#6169](https://togithub.com/psf/requests/issues/6169))
### [`v2.28.0`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2280-2022-06-09)
[Compare Source](https://togithub.com/psf/requests/compare/v2.27.1...v2.28.0)
**Deprecations**
- ⚠️ Requests has officially dropped support for Python 2.7. ⚠️ ([#6091](https://togithub.com/psf/requests/issues/6091))
- Requests has officially dropped support for Python 3.6 (including pypy3.6). ([#6091](https://togithub.com/psf/requests/issues/6091))
**Improvements**
- Wrap JSON parsing issues in Request's JSONDecodeError for payloads without
an encoding to make `json()` API consistent. ([#6097](https://togithub.com/psf/requests/issues/6097))
- Parse header components consistently, raising an InvalidHeader error in
all invalid cases. ([#6154](https://togithub.com/psf/requests/issues/6154))
- Added provisional 3.11 support with current beta build. ([#6155](https://togithub.com/psf/requests/issues/6155))
- Requests got a makeover and we decided to paint it black. ([#6095](https://togithub.com/psf/requests/issues/6095))
**Bugfixes**
- Fixed bug where setting `CURL_CA_BUNDLE` to an empty string would disable
cert verification. All Requests 2.x versions before 2.28.0 are affected. ([#6074](https://togithub.com/psf/requests/issues/6074))
- Fixed urllib3 exception leak, wrapping `urllib3.exceptions.SSLError` with
`requests.exceptions.SSLError` for `content` and `iter_content`. ([#6057](https://togithub.com/psf/requests/issues/6057))
- Fixed issue where invalid Windows registry entries caused proxy resolution
to raise an exception rather than ignoring the entry. ([#6149](https://togithub.com/psf/requests/issues/6149))
- Fixed issue where entire payload could be included in the error message for
JSONDecodeError. ([#6036](https://togithub.com/psf/requests/issues/6036))
### [`v2.27.1`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2271-2022-01-05)
[Compare Source](https://togithub.com/psf/requests/compare/v2.27.0...v2.27.1)
**Bugfixes**
- Fixed parsing issue that resulted in the `auth` component being
dropped from proxy URLs. ([#6028](https://togithub.com/psf/requests/issues/6028))
### [`v2.27.0`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2270-2022-01-03)
[Compare Source](https://togithub.com/psf/requests/compare/v2.26.0...v2.27.0)
**Improvements**
- Officially added support for Python 3.10. ([#5928](https://togithub.com/psf/requests/issues/5928))
- Added a `requests.exceptions.JSONDecodeError` to unify JSON exceptions between
Python 2 and 3. This gets raised in the `response.json()` method, and is
backwards compatible as it inherits from previously thrown exceptions.
Can be caught from `requests.exceptions.RequestException` as well. ([#5856](https://togithub.com/psf/requests/issues/5856))
- Improved error text for misnamed `InvalidSchema` and `MissingSchema`
exceptions. This is a temporary fix until exceptions can be renamed
(Schema->Scheme). ([#6017](https://togithub.com/psf/requests/issues/6017))
- Improved proxy parsing for proxy URLs missing a scheme. This will address
recent changes to `urlparse` in Python 3.9+. ([#5917](https://togithub.com/psf/requests/issues/5917))
**Bugfixes**
- Fixed defect in `extract_zipped_paths` which could result in an infinite loop
for some paths. ([#5851](https://togithub.com/psf/requests/issues/5851))
- Fixed handling for `AttributeError` when calculating length of files obtained
by `Tarfile.extractfile()`. ([#5239](https://togithub.com/psf/requests/issues/5239))
- Fixed urllib3 exception leak, wrapping `urllib3.exceptions.InvalidHeader` with
`requests.exceptions.InvalidHeader`. ([#5914](https://togithub.com/psf/requests/issues/5914))
- Fixed bug where two Host headers were sent for chunked requests. ([#5391](https://togithub.com/psf/requests/issues/5391))
- Fixed regression in Requests 2.26.0 where `Proxy-Authorization` was
incorrectly stripped from all requests sent with `Session.send`. ([#5924](https://togithub.com/psf/requests/issues/5924))
- Fixed performance regression in 2.26.0 for hosts with a large number of
proxies available in the environment. ([#5924](https://togithub.com/psf/requests/issues/5924))
- Fixed idna exception leak, wrapping `UnicodeError` with
`requests.exceptions.InvalidURL` for URLs with a leading dot (.) in the
domain. ([#5414](https://togithub.com/psf/requests/issues/5414))
**Deprecations**
- Requests support for Python 2.7 and 3.6 will be ending in 2022. While we
don't have exact dates, Requests 2.27.x is likely to be the last release
series providing support.
Configuration
📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
[ ] If you want to rebase/retry this PR, check this box
This PR has been generated by Mend Renovate. View repository job log here.
This PR contains the following updates:
==2.26.0
->==2.31.0
GitHub Vulnerability Alerts
CVE-2023-32681
Impact
Since Requests v2.3.0, Requests has been vulnerable to potentially leaking
Proxy-Authorization
headers to destination servers, specifically during redirects to an HTTPS origin. This is a product of howrebuild_proxies
is used to recompute and reattach theProxy-Authorization
header to requests when redirected. Note this behavior has only been observed to affect proxied requests when credentials are supplied in the URL user information component (e.g.https://username:password@proxy:8080
).Current vulnerable behavior(s):
For HTTP connections sent through the proxy, the proxy will identify the header in the request itself and remove it prior to forwarding to the destination server. However when sent over HTTPS, the
Proxy-Authorization
header must be sent in the CONNECT request as the proxy has no visibility into further tunneled requests. This results in Requests forwarding the header to the destination server unintentionally, allowing a malicious actor to potentially exfiltrate those credentials.The reason this currently works for HTTPS connections in Requests is the
Proxy-Authorization
header is also handled by urllib3 with our usage of the ProxyManager in adapters.py withproxy_manager_for
. This will compute the required proxy headers inproxy_headers
and pass them to the Proxy Manager, avoiding attaching them directly to the Request object. This will be our preferred option going forward for default usage.Patches
Starting in Requests v2.31.0, Requests will no longer attach this header to redirects with an HTTPS destination. This should have no negative impacts on the default behavior of the library as the proxy credentials are already properly being handled by urllib3's ProxyManager.
For users with custom adapters, this may be potentially breaking if you were already working around this behavior. The previous functionality of
rebuild_proxies
doesn't make sense in any case, so we would encourage any users impacted to migrate any handling of Proxy-Authorization directly into their custom adapter.Workarounds
For users who are not able to update Requests immediately, there is one potential workaround.
You may disable redirects by setting
allow_redirects
toFalse
on all calls through Requests top-level APIs. Note that if you're currently relying on redirect behaviors, you will need to capture the 3xx response codes and ensure a new request is made to the redirect destination.Credits
This vulnerability was discovered and disclosed by the following individuals.
Dennis Brinkrolf, Haxolot (https://haxolot.com/) Tobias Funke, (tobiasfunke93@gmail.com)
Release Notes
psf/requests (requests)
### [`v2.31.0`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2310-2023-05-22) [Compare Source](https://togithub.com/psf/requests/compare/v2.30.0...v2.31.0) **Security** - Versions of Requests between v2.3.0 and v2.30.0 are vulnerable to potential forwarding of `Proxy-Authorization` headers to destination servers when following HTTPS redirects. When proxies are defined with user info (https://user:pass@proxy:8080), Requests will construct a `Proxy-Authorization` header that is attached to the request to authenticate with the proxy. In cases where Requests receives a redirect response, it previously reattached the `Proxy-Authorization` header incorrectly, resulting in the value being sent through the tunneled connection to the destination server. Users who rely on defining their proxy credentials in the URL are *strongly* encouraged to upgrade to Requests 2.31.0+ to prevent unintentional leakage and rotate their proxy credentials once the change has been fully deployed. Users who do not use a proxy or do not supply their proxy credentials through the user information portion of their proxy URL are not subject to this vulnerability. Full details can be read in our [Github Security Advisory](https://togithub.com/psf/requests/security/advisories/GHSA-j8r2-6x86-q33q) and [CVE-2023-32681](https://nvd.nist.gov/vuln/detail/CVE-2023-32681). ### [`v2.30.0`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2300-2023-05-03) [Compare Source](https://togithub.com/psf/requests/compare/v2.29.0...v2.30.0) **Dependencies** - ⚠️ Added support for urllib3 2.0. ⚠️ This may contain minor breaking changes so we advise careful testing and reviewing https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html prior to upgrading. Users who wish to stay on urllib3 1.x can pin to `urllib3<2`. ### [`v2.29.0`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2290-2023-04-26) [Compare Source](https://togithub.com/psf/requests/compare/v2.28.2...v2.29.0) **Improvements** - Requests now defers chunked requests to the urllib3 implementation to improve standardization. ([#6226](https://togithub.com/psf/requests/issues/6226)) - Requests relaxes header component requirements to support bytes/str subclasses. ([#6356](https://togithub.com/psf/requests/issues/6356)) ### [`v2.28.2`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2282-2023-01-12) [Compare Source](https://togithub.com/psf/requests/compare/v2.28.1...v2.28.2) **Dependencies** - Requests now supports charset_normalizer 3.x. ([#6261](https://togithub.com/psf/requests/issues/6261)) **Bugfixes** - Updated MissingSchema exception to suggest https scheme rather than http. ([#6188](https://togithub.com/psf/requests/issues/6188)) ### [`v2.28.1`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2281-2022-06-29) [Compare Source](https://togithub.com/psf/requests/compare/v2.28.0...v2.28.1) **Improvements** - Speed optimization in `iter_content` with transition to `yield from`. ([#6170](https://togithub.com/psf/requests/issues/6170)) **Dependencies** - Added support for chardet 5.0.0 ([#6179](https://togithub.com/psf/requests/issues/6179)) - Added support for charset-normalizer 2.1.0 ([#6169](https://togithub.com/psf/requests/issues/6169)) ### [`v2.28.0`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2280-2022-06-09) [Compare Source](https://togithub.com/psf/requests/compare/v2.27.1...v2.28.0) **Deprecations** - ⚠️ Requests has officially dropped support for Python 2.7. ⚠️ ([#6091](https://togithub.com/psf/requests/issues/6091)) - Requests has officially dropped support for Python 3.6 (including pypy3.6). ([#6091](https://togithub.com/psf/requests/issues/6091)) **Improvements** - Wrap JSON parsing issues in Request's JSONDecodeError for payloads without an encoding to make `json()` API consistent. ([#6097](https://togithub.com/psf/requests/issues/6097)) - Parse header components consistently, raising an InvalidHeader error in all invalid cases. ([#6154](https://togithub.com/psf/requests/issues/6154)) - Added provisional 3.11 support with current beta build. ([#6155](https://togithub.com/psf/requests/issues/6155)) - Requests got a makeover and we decided to paint it black. ([#6095](https://togithub.com/psf/requests/issues/6095)) **Bugfixes** - Fixed bug where setting `CURL_CA_BUNDLE` to an empty string would disable cert verification. All Requests 2.x versions before 2.28.0 are affected. ([#6074](https://togithub.com/psf/requests/issues/6074)) - Fixed urllib3 exception leak, wrapping `urllib3.exceptions.SSLError` with `requests.exceptions.SSLError` for `content` and `iter_content`. ([#6057](https://togithub.com/psf/requests/issues/6057)) - Fixed issue where invalid Windows registry entries caused proxy resolution to raise an exception rather than ignoring the entry. ([#6149](https://togithub.com/psf/requests/issues/6149)) - Fixed issue where entire payload could be included in the error message for JSONDecodeError. ([#6036](https://togithub.com/psf/requests/issues/6036)) ### [`v2.27.1`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2271-2022-01-05) [Compare Source](https://togithub.com/psf/requests/compare/v2.27.0...v2.27.1) **Bugfixes** - Fixed parsing issue that resulted in the `auth` component being dropped from proxy URLs. ([#6028](https://togithub.com/psf/requests/issues/6028)) ### [`v2.27.0`](https://togithub.com/psf/requests/blob/HEAD/HISTORY.md#2270-2022-01-03) [Compare Source](https://togithub.com/psf/requests/compare/v2.26.0...v2.27.0) **Improvements** - Officially added support for Python 3.10. ([#5928](https://togithub.com/psf/requests/issues/5928)) - Added a `requests.exceptions.JSONDecodeError` to unify JSON exceptions between Python 2 and 3. This gets raised in the `response.json()` method, and is backwards compatible as it inherits from previously thrown exceptions. Can be caught from `requests.exceptions.RequestException` as well. ([#5856](https://togithub.com/psf/requests/issues/5856)) - Improved error text for misnamed `InvalidSchema` and `MissingSchema` exceptions. This is a temporary fix until exceptions can be renamed (Schema->Scheme). ([#6017](https://togithub.com/psf/requests/issues/6017)) - Improved proxy parsing for proxy URLs missing a scheme. This will address recent changes to `urlparse` in Python 3.9+. ([#5917](https://togithub.com/psf/requests/issues/5917)) **Bugfixes** - Fixed defect in `extract_zipped_paths` which could result in an infinite loop for some paths. ([#5851](https://togithub.com/psf/requests/issues/5851)) - Fixed handling for `AttributeError` when calculating length of files obtained by `Tarfile.extractfile()`. ([#5239](https://togithub.com/psf/requests/issues/5239)) - Fixed urllib3 exception leak, wrapping `urllib3.exceptions.InvalidHeader` with `requests.exceptions.InvalidHeader`. ([#5914](https://togithub.com/psf/requests/issues/5914)) - Fixed bug where two Host headers were sent for chunked requests. ([#5391](https://togithub.com/psf/requests/issues/5391)) - Fixed regression in Requests 2.26.0 where `Proxy-Authorization` was incorrectly stripped from all requests sent with `Session.send`. ([#5924](https://togithub.com/psf/requests/issues/5924)) - Fixed performance regression in 2.26.0 for hosts with a large number of proxies available in the environment. ([#5924](https://togithub.com/psf/requests/issues/5924)) - Fixed idna exception leak, wrapping `UnicodeError` with `requests.exceptions.InvalidURL` for URLs with a leading dot (.) in the domain. ([#5414](https://togithub.com/psf/requests/issues/5414)) **Deprecations** - Requests support for Python 2.7 and 3.6 will be ending in 2022. While we don't have exact dates, Requests 2.27.x is likely to be the last release series providing support.Configuration
📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Mend Renovate. View repository job log here.