search

sopel-irc / sopel-extras

Broken Sopel modules you might pick up and fix if you want them. Probably faster to just rewrite, though
40 stars 57 forks source link

use requests instead of willie's web library for SNI support #58

Closed anarcat closed 6 years ago

anarcat commented 9 years ago

without requests, https://blog.matan.me/ would return an SSL verification error:

ssl.CertificateError: hostname 'blog.matan.me' doesn't match either of 'ssl2000.cloudflare.com', 'cloudflare.com', '*.cloudflare.com' (file "/usr/lib/python3.4/ssl.py", line 284, in match_hostname)

I had expected python3 to support SNI, but it doesn't seem to be the case.

you can also compare the output of:

gnutls-cli blog.matan.me
gnutls-cli --disable-sni blog.matan.me

...to reproduce the "non-SNI" problem from gnutls-cli.

tyrope commented 9 years ago

Instead of patching this per module, perhaps this support would be better added to willie.web itself? (CC: @elad661 , who knows more of willie.web and SSL than I do)

anarcat commented 9 years ago

well i started thinking that the web wrapper should just completely be dropped in favor of requests, personnally. but that's an extra dependency. i don't see what willie.web gets us over requests, to be honest.

this, then, would be similar to https://github.com/embolalia/willie/issues/658 (use the standard logging module instead of a custom framework).

elad661 commented 9 years ago

I've looked into requests and saw some security related problems I did not like.

And this is not like using logging, because logging is part of the standard library. Requests, however, is not.

anarcat commented 9 years ago

yeah well, requests is widely available, through debian packages, pip and the like. it's certainly not as standard as logging, but short of rewriting it from scratch, it seems like a good way to go.

this is the second thing I couldn't accomplish with the builtin willie.web library (the first one was JSON-encoded form parsing and submission).

all software have security issues. which ones did you find in requests? did you report them? how fast did they get fixed? those would be the standards by which I would judge that upstream, not the fact that there may be some security issues. :)

elad661 commented 9 years ago

Few months ago when I evaluated requests I've seen some security bugs filed in their bug tracker that has been ignored for many months. I don't know if they are still open.

anarcat commented 9 years ago

i see. well, looking at the issue queue now is impressive: 1212 closed, 79 open... they seem to have their thing together. the only security-related issue i could find after a quick perusal is https://github.com/kennethreitz/requests/issues/1857 - and it seems almost complete already.

anarcat commented 9 years ago

bump: is there any horizon where this could be fixed in core? if so, how, if not depending on requests?

it seems a little difficult to endeavor to fix all of the standard python lib https support in willie - this is something that requests did, and they are actively working with upstream to fix that.

i also didn't see a response about the actual security issues you are refering to.

since this is all "extras", why not simply merge this in, knowing that this is peculiar to this module?

dgw commented 6 years ago

For a while, web.get() has been using requests internally, but I think this is still going to get merged as an extension of sopel-irc/sopel#1316. The "extras" modules aren't core, but they'll need to stop using the deprecated web methods when they're removed all the same.