wbond / package_control

The Sublime Text package manager
https://packagecontrol.io
4.79k stars 814 forks source link

Handle redirects with urllib #1510

Closed rwols closed 3 years ago

rwols commented 3 years ago

close #1416

Tested as follows: in the ST console run:

import sys
pc=sys.modules["Package Control"]
downloader=pc.package_control.downloaders.UrlLibDownloader({"cache_length": 1, "debug": False, "timeout": 100, "user_agent": "Hello there general kenobi"})
content=downloader.download("https://github.com/sublimelsp/LSP-lua/releases/download/v0.0.4/LSP-lua_osx-x64.zip", "oops", 100, 1)
import os
os.makedirs(os.path.join(sublime.packages_path(), "User", "Downloads"), exist_ok=True)
fp=open(os.path.join(sublime.packages_path(), "User", "Downloads", "asdf.zip"), "wb")
fp.write(content)
fp.close()

It looks like the oscrypto backend as well as the WinINet backend handles redirect, judging from skimming through the code.

rwols commented 3 years ago

I'm not sure if this would also resolve #1502, as that issue asks for specific handling of http status codes. But, in the end I think all we need is redirect handling.

rwols commented 3 years ago

This is odd. When I try the above steps on the master branch it also downloads the file. But curl is telling me it is indeed a 302:

~ $ curl -s -I https://github.com/sublimelsp/LSP-lua/releases/download/v0.0.4/LSP-lua_osx-x64.zip
HTTP/1.1 302 Found
date: Sun, 18 Oct 2020 21:52:04 GMT
content-type: text/html; charset=utf-8
server: GitHub.com
status: 302 Found
vary: X-PJAX, Accept-Encoding, Accept, X-Requested-With, Accept-Encoding
location: https://github-production-release-asset-2e65be.s3.amazonaws.com/294741644/86264e00-0572-11eb-808d-1533dc81d34c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20201018%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20201018T215204Z&X-Amz-Expires=300&X-Amz-Signature=bbc8f0e34cea9327707609862508fe93e6be34b8fbcbf411c5efd76c71e2332f&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=294741644&response-content-disposition=attachment%3B%20filename%3DLSP-lua_osx-x64.zip&response-content-type=application%2Foctet-stream
cache-control: no-cache
strict-transport-security: max-age=31536000; includeSubdomains; preload
x-frame-options: deny
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
referrer-policy: no-referrer-when-downgrade
expect-ct: max-age=2592000, report-uri="https://api.github.com/_private/browser/errors"
content-security-policy: default-src 'none'; base-uri 'self'; block-all-mixed-content; connect-src 'self' uploads.github.com www.githubstatus.com collector.githubapp.com api.github.com www.google-analytics.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com cdn.optimizely.com logx.optimizely.com/v1/events wss://alive.github.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com; frame-ancestors 'none'; frame-src render.githubusercontent.com; img-src 'self' data: github.githubassets.com identicons.github.com collector.githubapp.com github-cloud.s3.amazonaws.com *.githubusercontent.com; manifest-src 'self'; media-src 'none'; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; worker-src github.com/socket-worker.js gist.github.com/socket-worker.js
Set-Cookie: _gh_sess=B3xFdbQgHgyQ0IW6MThERKUFpb0XOzWzBQ0%2FZYNRk4cxrJHDV27fa%2BWt8o78woNp92P6%2F%2F%2F%2BNHW0mwUx260dlQRz1MFGI7buy2n348fPqmeDowzOtMiG9QhCg6lo%2B1%2B6oZGfvrvEikai7ituXsjbB05YgScvggZE7hyqKu5yTQJIJepHkvWY3Su5lyFBkCL7ChGLuyXAJxH37xq6mBksCUnMYWrl7egygMf089pValVWW2sC5HpkRNhYDeFDqR86sV36W5Bmv8jRxrUR0Mb7jA%3D%3D--%2Ff6cjD%2FeQj6TJtxR--T1oqNE1jaGlWsQy5n6q5yg%3D%3D; Path=/; HttpOnly; Secure; SameSite=Lax
Set-Cookie: _octo=GH1.1.645615566.1603057980; Path=/; Domain=github.com; Expires=Mon, 18 Oct 2021 21:53:00 GMT; Secure; SameSite=Lax
Set-Cookie: logged_in=no; Path=/; Domain=github.com; Expires=Mon, 18 Oct 2021 21:53:00 GMT; HttpOnly; Secure; SameSite=Lax
Content-Length: 645
X-GitHub-Request-Id: C0FB:133E:BD93F45:118E7409:5F8CB93C
rwols commented 3 years ago

I am not able to request a review. So, I'm asking via this comment for feedback, @wbond