Closed jayvdb closed 4 years ago
Even with the following patch, the same redirect occurs
diff --git a/pypidb/_pypi.py b/pypidb/_pypi.py
index c8de722..77142b6 100644
--- a/pypidb/_pypi.py
+++ b/pypidb/_pypi.py
@@ -57,12 +57,13 @@ social_definitions.update(
)
_DEFAULT_HEADERS = {
- "User-Agent": UserAgent().google,
- "Accept": "text/html,text/plain,application/*;q=0.8,text/*;q=0.5",
+ "User-Agent": UserAgent().firefox,
+ "Accept": "text/html,text/plain,application/xhtml+xml;q=0.9,application/xml;q=0.9,application/*;q=0.8,text/*;q=0.5,*/*;q=0.4",
"Accept-Encoding": "br,gzip;q=0.9,deflate;q=0.8"
if brotli
else "gzip,deflate;q=0.9",
"Accept-Language": "en-US,en;q=0.9",
+ "Upgrade-Insecure-Requests": "1",
}
The problem may be in https-everywhere-py, which may be dropped those headers during its internal logic which is doing a HEAD before the GET.
Also causes another failure:
"bloom": "https://github.com/ros/catkin"
Oddly, now in Chrome & Firefox, http://wiki.ros.org/catkin_pkg redirects to https://status.ros.org/ , but https://wiki.ros.org/catkin_pkg does not. This seems to be a bug upstream.
Created a topic at https://discourse.ros.org/, but it has gone into moderation.
It appears to be fixed. Without any notice on https://status.ros.org/ to indicate there was a failure.
Topic https://discourse.ros.org/t/wiki-ros-org-redirects-to-status-ros-org/13758 provides an explanation for the temporary failures.
http://wiki.ros.org/catkin_pkg in firefox loads normally, and the SCM is found on the page.
However in pypidb, it is redirected to https://status.ros.org/
Breaking catkin_pkg & rospkg & rosinstall (via fedora dataset) & rosinstall-generator & wstool and likely other ROS packages.