virtUOS / kiwi

A simple and configurable portal to proxy to llms.
https://virtuos.github.io/kiwi/
GNU Affero General Public License v3.0
2 stars 3 forks source link

Add environment variable for mapping user to model #35

Closed dennis531 closed 3 months ago

dennis531 commented 4 months ago

Add the envirnment variable USER_MODELS allowing to assign a different model to a user. The value consists of a simple json object with key-value pairs. The key contains the username and the value the configured model for the corresponding user.

Example:

USER_MODELS: '{"username1":"gpt-3.5","username2":"gpt-4"}'
Odrec commented 4 months ago

This is a good idea but I think the roles could be something like '{"username1":"restricted","username2":"open"}'

So the usernames with restricted role can only have access to the restricted model, in this case gpt-3.5, while users with open role can access and choose between gpt-3.5 or gpt-4o (in this case the most advanced model).

What do you think? In this case a dropdown widget to choose the model should be added.

This will allow in the future to adjust the access also to other models based on the level of access of the user.

So maybe have something like:

USER_ROLES: '{"username1":"restricted","username2":"open"}' MODELS_PER_ROLE: '{"restricted":["gpt-3.5", "gpt-3.5-turbo"],"open":["gpt-3.5", "gpt-3.5-turbo", "gpt-4o"]}'

Odrec commented 4 months ago

Another thing is that Rüdiger mentioned that it will be useful to have permissions per Fachbereich and not only per user. @yesidc said he made a modification to the LDAP code to return this information so we might want to consider roles per user or per Fachbereich.

So maybe something like:

ROLES: '{
"default":"restricted",
"faculties": {"Fachbereich 1":"open"}
"usernames":{"username1":"open", "username3":"open"}
}'

But maybe this could be added in another PR.

dennis531 commented 3 months ago

This is a good idea but I think the roles could be something like '{"username1":"restricted","username2":"open"}'

So the usernames with restricted role can only have access to the restricted model, in this case gpt-3.5, while users with open role can access and choose between gpt-3.5 or gpt-4o (in this case the most advanced model).

What do you think? In this case a dropdown widget to choose the model should be added.

This will allow in the future to adjust the access also to other models based on the level of access of the user.

So maybe have something like:

USER_ROLES: '{"username1":"restricted","username2":"open"}' MODELS_PER_ROLE: '{"restricted":["gpt-3.5", "gpt-3.5-turbo"],"open":["gpt-3.5", "gpt-3.5-turbo", "gpt-4o"]}'

I like the idea that the user can choose a model based on his role. I will implement this feature.

Another thing is that Rüdiger mentioned that it will be useful to have permissions per Fachbereich and not only per user. @yesidc said he made a modification to the LDAP code to return this information so we might want to consider roles per user or per Fachbereich.

I'll see if I can integrate it here. Otherwise I would also suggest a spearate PR.

yesidc commented 3 months ago

I just made a small change to the current script (for future reference, if needed)

from ldap3 import Server, Connection, ALL, AUTO_BIND_NO_TLS, Tls, ALL_ATTRIBUTES
  conn.search(
        ldap_base_dn,
        ldap_search_filter.format(username=username),
        attributes=[ALL_ATTRIBUTES])
dennis531 commented 3 months ago

I added the following environment variables:

Odrec commented 3 months ago

I added the following environment variables:

  • OPENAI_DEFAULT_MODEL: Replaces the old variable OPENAI_MODEL. This model will be used if no role or role models are configured.
  • USER_ROLES: Determines the role of a user.
  • MODELS_PER_ROLE: Assigns accessible models to a individual roles.

Great! LGTM :) So I'll merge this and do some tests