CURocketEngineering / firefly_chi-2021

Suite of tools/scripts for on-board telemetry and functioning avionics.
MIT License
6 stars 1 forks source link

"Mulithreading" #6

Closed HarrisonHall closed 4 years ago

HarrisonHall commented 4 years ago

Python isn't designed to support multithreading on multiple cores due to the global interpreter lock. Since the system will need to utilize shared memory, multiprocessing isn't a straightforward option.

  1. The simplest solution involves using python threading while acknowledging that this will still be on the same core. The advantage to this is that it keeps development simple (a producer-consumer model is used for data) and as soon as python is updated, the code will automatically be enhanced.

  2. A more advanced solution is to spawn the other processes as daemons and communicate over sockets. This means that we would have to keep track of raw data in SharedMemory along with a communication overhead. This would also take much more time to get right.

I would elect for option 1, a simple example can be found here. I would suggest utilizing a thread for the main loop (consumer), the data collection loop (producer), telemetry (consumer and producer), and logging (consumer). Alternatively logging can be split into 2 threads for flight-required logging and auxiliary data logging. The advantage of 4 threads is that in the case that GIL becomes available on all cores, the program is now optimized.

JackyFWong commented 4 years ago

As someone who has used both Python threading and concurrent.futures libraries, I vote for option 1 as well. Multi-threading is easy to do and should work at least well on a Pi. I personally used the producer-consumer relationship with the latter library, so I might recommend that.