search

ptpb / pb

pb is a formerly-lightweight pastebin and url shortener
Other
549 stars 52 forks source link

post request header storage and replay #243

Closed eddyb closed 5 years ago

eddyb commented 5 years ago

AFAIK, both of these are required to display a .svgz (gzip-compressed SVG) file:

Content-Type: image/svg+xml
Content-Encoding: gzip

But currently only the first header is sent, so the browser tries to interpret the file as an uncompressed SVG file, and fails. My testcase is https://ptpb.pw/9sCt.svgz.

See also https://github.com/h5bp/server-configs-nginx/issues/86 for another example where this kind of issue cropped up.

buhman commented 5 years ago

Content-Encoding: gzip

This isn't really a content-type/"doesn't work" issue, because there's no such thing as "image/svg+xml+gzip", which is why .svgz doesn't do anything useful.

I'd also like to point out that browser support for svg interpretation/rendering and browser support for gzip decompression are completely separate features. Content-Encoding: gzip could be used with basically ~anything, including gzip'ed plaintext, html, png, webm, etc...

If this feature were implemented, it would be more of a "store and replay POST headers on GET"; something like:

$ curl -F c=@my.svgz -H "Content-Encoding: gzip" -H "Content-Type: image/svg+xml" https://ptpb.pw
$ curl -D- https://ptpb.pw/my.extension-would-be-ignored-in-this-case
...
Content-Type: image/svg+xml
Content-Encoding: gzip
...
buhman commented 5 years ago

This is something I've thought pb has needed for awhile, but nobody ever requested it or presented a meaningful use-case.

buhman commented 5 years ago

implemented in: a8a1706693f46451b6b6e48ed79349b84485b43d deployed via: fb0d969a-f779-4411-b634-1ddd7aa8292d

Example:

➜  ~ curl -F c=@- -H 'x-foo-bar: spam' https://ptpb.pw <<< gook2134
date: 2019-01-07T03:13:36.533698+00:00
digest: 93c1103b89d5e04ab2f4a21601f16a9df53f7289
long: AJPBEDuJ1eBKsvSiFgHxap31P3KJ
short: P3KJ
size: 9
status: created
url: https://ptpb.pw/P3KJ
uuid: 9c54b52e-1e79-4b7a-b0e9-2c004088d7da
➜  ~ curl https://ptpb.pw/P3KJ
gook2134
➜  ~ curl -D- https://ptpb.pw/P3KJ
HTTP/1.1 200 OK
Server: openresty/1.13.6.2
Date: Mon, 07 Jan 2019 03:13:43 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 9
Connection: keep-alive
x-foo-bar: spam
x-varnish: 32774
ETag: "paste-93c1103b89d5e04ab2f4a21601f16a9df53f7289"
Cache-Control: public, max-age=43200
X-Varnish: 13 11
Age: 2
Via: 1.1 varnish (Varnish/6.0)
Accept-Ranges: bytes

gook2134

In your case, you want to add -H 'content-encoding: gzip' to your regular curl arguments.

While I think it would be nice to accept content-type in this way as well, due to past technical debt actually doing this isn't easy.

buhman commented 5 years ago

I also re-uploaded https://ptpb.pw/9sCt.svg to demonstrate:

curl -F c=@test.svg -H 'content-encoding: gzip' https://ptpb.pw

There is no difference between https://ptpb.pw/9sCt.svg and https://ptpb.pw/9sCt.svgz, both in the response from pb, and the resulting behavior from browsers.

buhman commented 5 years ago

As another example, here's a png with content-disposition: attachment (will trigger download dialog instead of rendering):

https://ptpb.pw/0OAB.png

buhman commented 5 years ago

And, just for fun, here is a gzip'ed webm:

$ TheFatRat\ -\ MAYDAY\ feat.\ Laura\ Brehm-DT61L8hbbJ4.webm | gzip | curl -F c=@- -H 'content-encoding: gzip' https://ptpb.pw
date: 2019-01-07T03:37:53.337885+00:00
digest: af5501bf58f25d07410aa7731dca80a0088d7e06
long: AK9VAb9Y8l0HQQqncx3KgKAIjX4G
short: jX4G
size: 22982644
status: created
url: https://ptpb.pw/jX4G
uuid: 7a29ec60-dc00-47d8-836f-02359b80bdb5
$ wc TheFatRat\ -\ MAYDAY\ feat.\ Laura\ Brehm-DT61L8hbbJ4.webm
   90609   512571 23030577 TheFatRat - MAYDAY feat. Laura Brehm-DT61L8hbbJ4.webm

amusing reduction 23030577 -> 22982644, what a savings

buhman commented 5 years ago

Interestingly, varnish is trying to be "smart", which confounds testing:

$ curl -s -D- https://ptpb.pw/McM6 | wc 
   1298   12249  109818
$ curl -H 'Accept-Encoding: gzip' -s -D- https://ptpb.pw/McM6 | wc 
    121     571   22832

Fixed in 57eca7f1-6e93-43f6-92ed-ac91434c8851.

buhman commented 5 years ago 
$ curl -s https://ptpb.pw/jX4G | mpv /dev/stdin
Playing: /dev/stdin
Failed to recognize file format.

Exiting... (Errors when loading file)
$ curl -s https://ptpb.pw/jX4G | gzip -d | mpv /dev/stdin
Playing: /dev/stdin
[mkv] SeekHead position beyond end of file - incomplete file?
 (+) Video --vid=1 (*) (vp9 1920x1080 29.970fps)
 (+) Audio --aid=1 --alang=eng (*) (vorbis 2ch 44100Hz)
AO: [pulse] 44100Hz stereo 2ch float
VO: [gpu] 1920x1080 yuv420p
AV: 00:00:01 / 00:04:07 (0%) A-V:  0.000 Cache: 124s+15MB

Exiting... (Quit)

And indeed, https://ptpb.pw/jX4G.webm works in firefox/chromium automatically.