JulianFP / project-W

A self-hostable platform on which users can create transcripts of their audio files (speech-to-text) using Whisper AI
GNU Affero General Public License v3.0
2 stars 0 forks source link

Make backend aware of the runners hardware capabilities #18

Open JulianFP opened 6 months ago

JulianFP commented 6 months ago

Currently the backend will treat all runners equally. If your runners run on very similar or equal hardware then this is not a problem, however if that is not the case it very much is: If for example if one runner runs on a NVIDIA A100 GPU, and another on an old CPU, then the backend might still send jobs to the second runner even if the first runner would also be free.

Proposition: Add some basic benchmark score (maybe even just a score that the admins defines when creating the runner) that tells the backend how powerful a runner is (e.g. a number between 0 and 100). When the backend then assigns jobs to runners it could prefer runners with higher scores over runners with lower ones (among the available runners for this job).