Automatically use free GPU

[anne17]

Stanza väljer alltid GPU 0 automatiskt, men om en maskin har flera så borde den automatiskt välja den som är minst upptagen. Man kan styra vilken GPU som ska användas med flaggan CUDA_VISIBLE_DEVICES (t.ex. CUDA_VISIBLE_DEVICES=1 sparv run).

Kanske skulle man kunna bygga in detta i Sparv att den vid körning av en annoterare som använder GPU, så kollar den först om det finns flera GPU:er, vilken som är ledigast, och sätter miljövariabeln automatiskt för just det jobbet. (Martins idé)

[MartinHammarstedt]

I've now implemented the following:

• When Sparv first starts, it checks the available GPUs and orders them by amount of free memory in the CUDA_VISIBLE_DEVICES environment variable. This means that any module that simply uses the first available GPU, will now use the one that had most free memory at the startup of Sparv.
• At runtime, the utility function util.system.gpus() will now give a list of GPUs, ordered by amount of free memory in descending order. This can be used by modules to select the GPU with most free memory at that moment.
• Stanza now uses the aforementioned function to select the most appropriate GPU, meaning that it should no longer be a problem running as many parallel Stanza processes as there are GPUs.

We could update CUDA_VISIBLE_DEVICES every time a module runs, to make modules always select the currently most appropriate GPU without needing to explicitly do so themselves, but since we don't know which modules use GPUs, that would mean a lot of unnecessary GPU checking for every single job item. I think the above solution is a good compromise.