python / cpython

The Python programming language
https://www.python.org
Other
63.07k stars 30.2k forks source link

urllib proxy interface is too limited #52330

Open ronaldoussoren opened 14 years ago

ronaldoussoren commented 14 years ago
BPO 8083
Nosy @ronaldoussoren, @orsenthil

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields: ```python assignee = 'https://github.com/orsenthil' closed_at = None created_at = labels = ['type-feature', 'library'] title = 'urllib proxy interface is too limited' updated_at = user = 'https://github.com/ronaldoussoren' ``` bugs.python.org fields: ```python activity = actor = 'numerodix' assignee = 'orsenthil' closed = False closed_date = None closer = None components = ['Library (Lib)'] creation = creator = 'ronaldoussoren' dependencies = [] files = [] hgrepos = [] issue_num = 8083 keywords = [] message_count = 2.0 messages = ['100573', '100714'] nosy_count = 6.0 nosy_names = ['ronaldoussoren', 'orsenthil', 'Dominique.Leuenberger', 'tshepang', 'Ankur.Ankan', 'numerodix'] pr_nums = [] priority = 'normal' resolution = None stage = 'needs patch' status = 'open' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue8083' versions = ['Python 3.4'] ```

ronaldoussoren commented 14 years ago

The proxy support code in urllib is imho too specific and cannot easily support dynamic proxy configuration using a proxy.pac file.

That is, the code assumes that there is at most one proxy per protocol, while this is not quite true in practice. A proxy.pac can refer different URLs to different proxies (or exclude some from proxying).

This could be solved by adding a new proxy hook to urllib, something like:

def getproxies_for_url(url) -> [info, ...]

That is, the argument is the string for an URL, the result is a list of proxies that can be used to fetch the URL.

This hook would allow us to provide a hook on OSX that enables python scripts to use the system proxy settings, even when those are complex (such as by using proxy autoconfiguration).

4b17aa54-3576-4bf3-b56a-a1073f20d1c0 commented 14 years ago

I like the idea of having the proxy handler expanded. In fact I suggest to base the idea on libproxy ( http://code.google.com/p/libproxy ) which is available on Linux / openSolaris / Windows and Mac (currently).

Libproxy queries the correct settings to be used from KDE and gnome, depending on the currently running session. A fallback is of course env.

the API of libproxy is very simple and python bindings are available.