Investigating Python concurrent tasks execution alternatives

ringleschavez commented 3 years ago

Aspect	Detail
Summary	Checking the alternatives provide by Python to start a new task
Task Area	Investigation
Assignee	ringleschavez
Information	r.ingles.chavez@gmail.com
Prerequisites
Dependencies

Summary

Reviewing some books where Python multi-processing approaches are described, focusing on the pros and cons inherent to them when are used for I/O-Bound processing and CPU-Bound processing.

Tasks

[x] Reviewing the current test cases scripts where new processes or threads are involved.
[x] Defining the scope of the processes, threads and co-routines on the existent test case scripts.
[ ] Reviewing Python's bibliography in order to get a big picture about the alternatives provided by the current release of Python.
[ ] Defining which multi-processing approach will be used for which processing task, namely I/O-Bound producer, I/O-Bound consumer and CPU-Bound.

Requirements

Checking the approaches provided by Python related to launching processing tasks.

Acceptance criteria

Definition of which approach is a better choice for the identified use cases where multiple processes, threads or co-routines are launched.

ringleschavez commented 3 years ago

Selected bibliography:

Effective Python: 90 Specific Ways to Write Better Python by Brett Slatkin, 2019
Advanced Python Development: Using Powerful Language Features in Real-World Applications by Matthew Wilkes (Autor), 2020
Advanced Python Programming: Build high performance, concurrent, and multi-threaded apps with Python using proven design patterns by Dr. Gabriele Lanaro (Autor), Quan Nguyen (Autor), Sakis Kasampalis (Autor), 2019
High Performance Python: Practical Performant Programming for Humans by Micha Gorelick (Autor), Ian Ozsvald (Autor), 2020

ringleschavez commented 3 years ago

Additional Bibliography (to be checked):

Python in Practice: Create Better Programs Using Concurrency, Libraries, and Pat: Create Better Programs Using Concurrency, Libraries, and Patterns (Developer's Library) by Mark Summerfield, 2013
Python: Master the Art of Design Patterns by Dusty Phillips, Chetan Giridhar, Sakis Kasampalis, 2016
Python High Performance - Second Edition: Build high-performing, concurrent, and distributed applications by Gabriele Lanaro (Autor), 2017
Advanced Guide to Python 3 Programming by John Hunt, 2019
Expert Python Programming: Become a master in Python by learning coding best practices and advanced programming concepts in Python 3.7 by Michal Jaworski, Tarek Ziade, 2019
Python Cookbook: Recipes for Mastering Python 3 Paperback by David Beazley, Brian K. Jones, 2013

w-klijn commented 3 years ago

Investigation has been done

multiscale-cosim / EBRAINS-cosim