Closed zuisong closed 3 months ago
I'm wondering if we should change all calls to HeaderValue.to_str()
to HeaderValue.to_utf8_str()
.
I think it's a good idea.
For example, this will allow us to download filenames that support UTF-8 encoding. https://github.com/ducaale/xh/blob/ca629c803f78834a32994f6ca6754cd6fc896e63/src/download.rs#L34
Officially you're not supposed to do that so we should decide on a case-by-case basis.
For filenames we can actually pass the bytes through directly depending on the platform. But we should check what different servers and clients do.
Yes, you're right.
I decided to use UTF-8 encoding to decode the CONTENT_DISPOSITION response header, which is very useful for downloading files with Unicode names. And Chrome implements it the same way.
before
❯ xh httpbingo.org/response-headers 'content-disposition==attachment; filename="🥰.json"' -d
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: close
Content-Disposition: "attachment; filename=\"\xf0\x9f\xa5\xb0.json\""
Content-Length: 76
Content-Type: application/json; charset=utf-8
Date: Fri, 21 Jun 2024 16:07:28 GMT
Fly-Request-Id: 01J0XRV0MHF1VBYPJY576KHWSM-sjc
Server: Fly/ebd3372a (2024-06-19)
Via: 1.1 fly.io
Downloading 76 B to "response-headers.json-3"
Done. 76 B in 0.00108s (68.80 KiB/s)
after
❯ /Users/chen/.rust-target/debug/xh httpbingo.org/response-headers 'content-disposition==attachment; filename="🥰.json"' -d
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: close
Content-Disposition: attachment; filename="🥰.json"
Content-Length: 76
Content-Type: application/json; charset=utf-8
Date: Fri, 21 Jun 2024 16:06:44 GMT
Fly-Request-Id: 01J0XRSP2WEE21V3XAHNGBFCBY-sjc
Server: Fly/ebd3372a (2024-06-19)
Via: 1.1 fly.io
Downloading 76 B to "🥰.json"
Done. 76 B in 0.00144s (51.63 KiB/s)
For reference, this behavior is the same as the redirection behavior of reqwest.
see https://github.com/seanmonstar/reqwest/blob/29d4cff234b37065632512f002b9785700c51aa8/src/async_impl/client.rs#L2624
related issue: https://github.com/ducaale/xh/issues/323