Asynchronous programming in Python3.5+

[xxleyi]

Asynchronous programming in Python has become more and more popular lately.

Asynchronous programming. Python3.5+

[xxleyi]

Quick Recap:

• Sync: Blocking operations.
• Async: Non-blocking operations.
• Concurrency: Making progress together.
• Parallelism: Making progress in parallel.
• Parallelism implies Concurrency. But Concurrency doesn’t always mean Parallelism.

[xxleyi]

Python code can now be mainly executed in one of two worlds, synchronous or asynchronous. You should think of them as separate worlds with different libraries and styles of calls, but sharing variables and syntax.

[xxleyi]

Threads

A thread is the smallest unit of processing that can be performed in an OS.

Threads of a process can share the memory of global variables. If a global variable is changed in one thread, this change is valid for all threads.

In simple words, a thread is a sequence of such operations within a program that can be executed independently of other code.

Threads executing concurrently but can be executing in parallel — it depends on the system on which they are running.

Global Interpreter Lock (GIL)

First of all, GIL is a lock that must be taken before any access to Python (and this is not only the execution of Python code but also calls to the Python C API). In essence, GIL is a global semaphore that does not allow more than one thread to work simultaneously within an interpreter.

In fact, GIL in python makes the idea of ​​using threads for parallelism in computational problems(CPU-bound operations) useless.

On CPU-bound tasks, the program will not accelerate, but only slow down, because now the threads will have to halve the processor time. At the same time, the GIL I/O operation will not slow down, since before the system call the thread releases the GIL.

On this sad note, you can come to the conclusion that threads will be enough for parallelizing tasks that are tied to I/O. But computing tasks should be run in separate processes.

[xxleyi]

Processes

Like threads, processes are always executed concurrently, but they can also be executed in parallel, depending on the presence of the hardware component.

The calculation operation executed faster than threaded implementation because now we are not stuck in capturing GIL, but the I/O bound function took slightly more time because processes are more heavyweight than threads.

[xxleyi]

Asynchronous world

TODO

[xxleyi]

Making the Right Choice

• CPU-bound -> multiprocessing
• I/O-bound, fast I/O, Limited Number of Connections -> multithreading
• I/O-bound, slow I/O, many connections -> asyncio

[xxleyi]

Thread Synchronization Mechanisms in Python