search

run-ai / genv

GPU environment and cluster management with LLM support
https://www.genv.dev
GNU Affero General Public License v3.0
445 stars 19 forks source link

Feature request: support for per-user enforcement #50

Closed crichardson332 closed 4 months ago

crichardson332 commented 1 year ago

Hello, I'm in an academic AI lab that uses genv to manage access to our compute resources. The tool has been great so far, but one thing that would be really convenient for our lab is the ability to set different quotas/constraints for different users, e.g. setting a lower max_devices for certain students in our lab (we want PhD students to have access to more GPUs than undergrads, for example). Having a simple config file for every user that defines their quotas would suffice.

This kind of feature would be greatly appreciated. Thanks!

razrotenberg commented 1 year ago

hi @crichardson332 and thanks for reaching out! this sounds like a super relevant capability to add to Genv. I also think that it won't take long to implement.

just to make sure, are you using the command genv remote enforce from one of the machines (with admin privileges) and want to enforce the usage of GPUs by students that are using genv remote activate commands?

or your setup includes only a single machine and we are talking about genv enforce and genv activate (w/o remote)

razrotenberg commented 1 year ago

hi @crichardson332 I just released Genv 1.1.0 which includes the functionality you asked for. Although I agree that a configuration file is best for this case, it is not yet implemented in this way, and could be in a future version.

For now, you can pass the new --max-devices-for-user argument to genv enforce and genv remote enforce commands. Using this argument you can pass user-specific maximum values for users. It should be passed together with the existing --max-devices-per-user.

For example, by passing --max-devices-per-user 1 --max-devices-for-user john=3 paul=2, you enforce all users to use a single device at most, except for John and Paul which are allowed to use 3 and 2 respectively.

You can see more information here

Please let me know if this works for you or not. Waiting to hear from you, and thanks again for reaching out!

razrotenberg commented 11 months ago

hey @crichardson332 have you had the chance to test it?

anindyamaiti commented 4 months ago

I am using genv in a similar fashion in my lab (without remote). The per user enforcement does work as expected.

Thanks for this very useful tool!

razrotenberg commented 4 months ago

hey @anindyamaiti

thank you for taking the time and sharing your experience here and I'm happy to hear Genv helps you!