search

gchq / gaffer-tools

gaffer-tools is deprecated. Use https://github.com/gchq/gafferpy instead
Apache License 2.0
50 stars 29 forks source link

Gaffer API can't connect to services behind proxies using SSL #863

Closed sw96411 closed 3 years ago

sw96411 commented 3 years ago

GafferPy is written using urllib to do it's HTTP connections, which is a venerable old thing.

There's a limitation of urllib that stops it being able to connect to HTTPS services that are behind a proxy. These days, that describes quite a lot of corporate environments and therefore puts an unnecessary limit on who can take advantage of the python API for Gaffer.

Swapping to requests instead of URLLib would be a fairly straightforward way of resolving this issue - I've tried this out locally and it seems fine so I will tidy a few things up and hang a PR off this issue if that's OK? An alternative is to keep urllib and write a custom HTTPSOpener, but requests produces more readable code, with less custom elements to go wrong

d47853 commented 3 years ago

sounds good!

t92549 commented 3 years ago

Hi @sw96411, any updates on this? I can take this up if you would like

sw96411 commented 3 years ago

Sorry for leaving this hanging.

I have a solution (a subclass of GafferConnector) that works in a particular deployment of Gaffer but it's very opinionated about exactly how the proxy/SSL environment is configured, and therefore not suited to a PR into this project, where we need something that will work much more generically.

I spent an afternoon trying to make my environment-specific code more generic but ended up tying myself in knots, and then life and other requirements intervened.

I'll send you a private e-mail with the code I did end up using - feel free to take that work forward or to reach out for a chat as appropriate,

t92549 commented 3 years ago

Excellent, that would be really helpful. Will try and take that forward to something generic that can solve this issue