search

wbond / package_control

The Sublime Text package manager
https://packagecontrol.io
4.79k stars 814 forks source link

Thoughts on asyncio #1509

Closed rwols closed 1 year ago

rwols commented 3 years ago

I would like to see this ST package utilize asyncio to its fullest for py3.8 in order to decrease the abundant thread usage in packages today. Are you interested in someone starting work on this aspect?

Some initial problems:

wbond commented 3 years ago

There would have to be a bit of investigative work done here. I think it would involving seeing if there are Windows async APIs for WinINet and oscrypto’s TLS layer would need to have async support added, or we’d have to roll back adding that.

I’m not familiar with how you’d make subprocess interaction with curl and wget async.

My hunch is that this would be a rather significant amount of work.

wbond commented 3 years ago

The other, bigger issue is that ST3 support wouldn’t be going away any time soon, so we’d have to have threading around anyway.

rwols commented 3 years ago

Why were asn1crypto and oscrypto included as dependencies? I would have to understand this history better...

wbond commented 3 years ago

asn1crypto and oscrypto were added to support TLS certs from the OS rather than PC trying to maintain its own PKI.

wbond commented 3 years ago

It also means that we support local CA certs required by middle boxes, etc.

rwols commented 3 years ago

I've been skimming through the source of oscrypto for the past few hours. Quite the journey :) Yes, the oscrypto.TLSSocket would have to be "asyncified" for all this to work. I haven't put any time into WinINet yet.

Maybe related: doing a DNS lookup with asyncio.getaddrinfo secretly creates a thread. So, if we really want to have zero thread overhead we have to use something like the aiodns library (a python wrapper around the c-ares library). Perhaps it would be good to focus on getting that shaped into an ST 3.8 dep first.

wbond commented 3 years ago

The WinINet stuff is actually part of Package Control and not oscrypto since it is a high-level HTTP library, whereas oscrypto provide a low-level TLS socket.

deathaxe commented 1 year ago

With Package Control being intented to support ST3, asyncio is ruled out.