dandroid126
commented
1 year ago I'm not super experienced in multithreading, but just spitballing on the architecture here:
The main thread is responsible for pulling the data from the API then spawning the worker threads to update the games. It is no longer responsible for creating/editing the posts. It pulls the data from the API, parses them into objects, and gets a list of games (same as it does now). For each game in the list, the main thread spawns a worker thread to do the work that is happening in the main loop now, but in a thread instead. Back in the main thread, once all the child threads have been created, sleep for the wait time and then poll the API again.
Things to consider:
What happens if a thread gets stuck? Best case scenario, that iteration through the loop gets dropped, and the game gets updated on a new thread. Yes, that thread will be held open until the timeout, but it will just be blocked and will not using any CPU resources. Worst case scenario, what if that iteration does go through, but after a significant delay, let's say because of a network issue. What happens if two threads are trying to edit the same post at the same time? Will that cause an exception thrown from pythorhead?
This will prevent all games from getting stuck when one game gets stuck from the unaddressed timeout issue in pythorhead. Also, just in general, I think it would be a good idea to have the games update independently of each other in the event some unforeseen issue holds the main thread up.