fragment not append on redirect

[stcioc]

In the following case (see detailed explanation and test case at http://www.webdbg.com/test/redir/fragment/):

• try to open a link that contains a fragment (say http://foo/#SomeInfo)
• the server sends a 302 Redir to http://bar

Expected Result

The expected result is that requests library follows the redirection and appends the original fragment from the link (e.g. it redirects to http://bar/#SomeInfo)

Actual Result

The redirect link is followed without the fragment (e.g. requests tries to open http://bar)

Reproduction Steps

import requests
response = requests.get("https://www.bayden.com/test/redir/fragment/redir-1.asp#/HereIsSomeInfo")
html = response.text
print(response.url)

output is "http://www.enhanceie.com/test/redir/fragment/final.asp" output should be "http://www.enhanceie.com/test/redir/fragment/final.asp#HereIsSomeInfo"

System Information

$ python -m requests.help

{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.2"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "100020bf"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": false
}

This command is only available on Requests v2.16.4 and greater. Otherwise, please provide some basic information about your system (Python version, operating system, &c).

[sigmavirus24]

This isn't how redirects work. The RFCs (3986, 7230, 7231, 7232, 7233, 7234, 7235) do not require or prescribe retaining the fragment. If you want us to retain the fragment you'll need to handle redirects yourself. The server is telling us what to redirect to and we're supposed to follow that.

Thanks for opening this issue to clarify.

[sigmavirus24]

Actually this was formalized in the rewrite of HTTP/1.1 (see also https://stackoverflow.com/questions/2286402/url-fragment-and-302-redirects)

[sigmavirus24]

This is a fantastic way to leak some data to an untrusted server, however, and is probably far from ideal. Oh well, we'll have to fix this.

[sigmavirus24]

In case that StackOverflow question goes away, here's the relevant portion of 7231: https://tools.ietf.org/html/rfc7231#section-7.1.2

[nateprewitt]

This should be resolved on master with #4452 and available in the next release. Thanks again for the report @stcioc!