Python 3 support

[chrysn]

pydtls currently does not support Python 3.

I've tried the trivial porting steps of running 2to3, and replaced a few str/bytes changes, but the largest chunk of work seems to be in dtls/patch.py where pydtls is accessing socket internals that changed (eg. _sock, but even if that was Python 2 idiom for accessing the fileno, changing socket(_sock=other._sock) to socket(fileno=other.fileno()) didn't get me far on the test suite).

Please consider adding support for current Python versions.

[notedit]

+1

[chrysn]

I've made some progress with crude Python 3 porting -- far from production quality (I'm completely disregarding anything pre-3 right now, introducing memory leaks to keep the originals from cloned socket objects from closing the clones at GC time, etc), but it might lead to insights on what's needed for a proper port (eg. that checking for sock.type == SOCK_DGRAM needs to be replaced with a getsockopt, because the type nowadays contains information on nonblockingness).

I'll publish the branch when it's good enough for a simple demo to run or I get stuck, but probably today.

[chrysn]

My python3 branch contains now a mess of changes that might be sufficient to get simple things running. IMO next steps are:

• pydtls project will need to decide on a Python 3 strategy -- whether six or 3to2 or something like that will be used; can't help there, I never maintained a works-for-both library
• Some changes are rather straightforward and (with some six-or-similar support) should also work on Python 2 (eg. replacing ".join([chr(x) for x in ...]) with bytes(...), skipping irrelevant tests on Python 3 or using getsockopt instead of .type to account for blocking information in Python 3's type field)
• Some changes need more consideration; especially, the sock._sock mechanism is completely different in Python 3 (the socket is the object itself and not its ._sock property; my workaround is happily creating identical socket objects and keeps the unused ones in an artificial memory leak to avoid closing), and needs to be re-thought by someone who actually understands how pydtls is supposed to work and doesn't just put together the bits that break under the unit test.

The above got me far enough to discover that I'll be blocked by https://bugs.python.org/issue19084 with pydtls (no way to use preshared keys (PSK) with Python's ssl). I'll stay subscribed to this bug in case anything around my patches is unclear, but unless I see a roadmap with the underlying PSK issue with ssl is resolved, I won't push this any further on my own.

[rbit]

Where are you encountering the PSK issue?

[chrysn]

On Tue, Apr 11, 2017 at 11:53:35AM -0700, Ray Brown wrote:

Where are you encountering the PSK issue?

I take this as a question for the application I have for PSK: the CoAP protocol (like HTTP but suitable for very constrained ("IoT") devices) uses DTLS 1.2 as its transport layer encryption, and the first widely available devices that support it out of the box use preshared keys for easy deployment (https://github.com/chrysn/aiocoap/issues/67).

[rbit]

I see. Thank you. What is the sequence of events that would require the PSK code path to be wired through the standard ssl module, as opposed to through PyDTLS into OpenSSL?

[chrysn]

I don't really need it wired through stdlib; it just seemed to me that the ciphers of pydtls all come from ssl, and that that would be the way to go. If the TLS_PSK_WITH_AES_128_CCM_8 cipher could be directly added to pydtls, that would save me quite some trouble :-)

[rbit]

PyDTLS imports the standard ssl module only for the use case that enables passing of datagram sockets to that module's public functions. This allows users to use standard ssl module syntax, semantics, and error mechanisms. For uses cases not requiring this, there is the SSLConnection interface (see the README). SSLConnection objects are used entirely independently of the standard ssl module.

My feeling about a Python 3 port is that it should be mostly independent of the standard ssl module - perhaps only dealing with it where performance demands (see the section Multi-thread Support in the README); patch.py should probably be cut entirely. It's not that upgrading the standard ssl module isn't a good thing - leveraging users' knowledge and familiarity with its conventions is powerful. However, upgrading the standard ssl module for datagram support requires reaching into its internal and undocumented implementation details, as you've probably noticed. Chasing the active code base of Python 3 in this way is probably not something most maintainers would want to sign up for. This does mean that the unit tests need to be adapted to use SSLConnection directly, though.

[hgarcia101685]

Is active development on this being done? I'd be willing to contribute wherever needed if that will help

[chrysn]

It appears that the fork at https://github.com/mobius-software-ltd/pyton3-dtls is what now populates the PyPI pytho3-dtls package.

It's all a bit unclear, because the package from the @mobius-software-ltd repo claims to be installable as pip install python3-dtls while the PyPI image says its home page is's yet another one (https://github.com/mcfreis/pydtls by @mcfreis but that hasn't seen the most recent updates PyPI has). Some clarification, both from in this repo (stating it's abandoned and on Python 2, and where to look at for successors) as well as on the forks on lineage (or picking a canonical repository at least) would be appreciated.

[hgarcia101685]

Thanks for the info @chrysn !