frostming / unearth

A utility to fetch and download python packages
https://unearth.readthedocs.io
MIT License
142 stars 18 forks source link

Handle 403, 404 as well as 401 #69

Closed logangrado closed 1 year ago

logangrado commented 1 year ago

Currently, if unearth receives a 401 when attempting to retrieve a package, it will automatically look for credentials in other locations, such as the ~/.netrc file. The relevant section of code is here

However, many private pypi artifactories will return 403 or even 404 (such as jfrog) if credentials are not provided. Currently, my group does not use PDM specifically for this reason, as it cannot automatically pick up credentials from ~/.netrc for our artifactories.

It would be a great addition to make a small update to the handling of credentials such that if one of these error codes was returned, that unearth would search for credentials and try again.

This change would be very small - essentially changing L290 here from if resp.status_code != 401:... to if resp.status_code not in [401, 403, 404]:... (and update a few names/comments)

I would be happy to make a PR to make this change if the maintainers are open to it.

frostming commented 1 year ago

handling 404 as 401 isn't a good idea. While it's good to have an option to extend the unauthorized status codes, and by default only 401 is handled.

logangrado commented 1 year ago

I agree, handling 404 the same as a 401 wouldn't be a good idea. It wouldn't be a great experience to be prompted for credentials from a 404.

However, the fact that many artifactories allow hiding URLs behind 404 (to ensure outsiders cannot determine names of packages, "hide existence of unauthorized resources" option) means it would be good to have some way to handle 404. as well

It is my understanding that 401's are currently handled as follows:

What if: