ArgLab / ArgLab_writing_observer

Writing Observer and Learning Observer: A system for monitoring learning process data, with an initial focus on writing process data from Google Docs.
GNU Affero General Public License v3.0
3 stars 2 forks source link

Distribute NLP Calls across GPUs. #57

Open DrLynch opened 1 year ago

DrLynch commented 1 year ago

Pending resolution of #46 we have to also find a way to make use of both GPUs on our system. Current single-GPU or no-GPU operation is viable up to a limit. However to make it actually workable across the board requires us to be able to dynamically distribute the calls. This means finding a good way to make the code recognize the presence of multiple GPUs; track the load per-GPU; and distribute calls as needed;.

All of this can be done but needs some better load management using standard engineering practice.