Managing Scale - Githubissues

tattle-made / DAU

MCA Tipline for Deepfakes

GNU General Public License v3.0

6 stars 0 forks source link

Managing Scale #33

Open dennyabrain opened 4 months ago

dennyabrain commented 4 months ago

Features

[x] #36
[x] #59
[ ] #103

dennyabrain commented 4 months ago

Continuing the exploration on Python's multiprocessing issue code. If we want to use multiprocessing on multicore nodes we will need some mechanism for these workers to

announce their availability (so they arent over allocated jobs)
take up jobs as they come and respond with success/failure appropriately