Move model selection to .env

[Torhamilton]

Either make gpt4 default or allow admin to set in .env. Users may override it with /model

[c0sogi]

I think you can change default model by changing LLMModels.gpt_3_5_turbo in the construct_default function, which is located in app\models\gpt_models.py.

    @classmethod
    def construct_default(
        cls,
        user_id: str,
        chat_room_id: str,
        gpt_model: LLMModels = LLMModels.gpt_3_5_turbo,
    ):
        return cls(
            user_gpt_profile=UserGptProfile(user_id=user_id, chat_room_id=chat_room_id),
            gpt_model=gpt_model,
        )

I allowed users to change model. If you don't want users to change llm model, just add few lines of validation code in changemodel function, which is located in app\utils\chatgpt\chatgpt_commands.py.

    @staticmethod
    @CommandResponse.send_message_and_stop
    async def changemodel(model: str, user_gpt_context: UserGptContext) -> str:
        """Change GPT model\n
        /changemodel <model>"""
        if model not in LLMModels._member_names_:
            return f"Model must be one of {', '.join(LLMModels._member_names_)}"
        llm_model: LLMModels = LLMModels._member_map_[model]  # type: ignore
        user_gpt_context.gpt_model = llm_model
        await ChatGptCacheManager.update_profile_and_model(user_gpt_context)
        return f"Model changed to {model}. Actual model: {llm_model.value.name}"

[Torhamilton]

Yes I did change the default version. However, updates will overwrite it.

[c0sogi]

Each chat has its own Gpt model. By default, the newly created chat will use the model defined in 'construct_default'. And yes, entering '/model' will overwrite the default, and will permanently change model of the chat. You can check model by entering '/test' in chat. If you want to make it default, just create new chat, and this chat uses default model, which is independent from other chats

[c0sogi]

The reason why your model wasn't changed as though you changed its default model, is that this previous model name was stored in Redis. So, when entering chat, the model you previosly set will be loaded into the chat.

If you want to change model of all chats, maybe changing the code of reading context. Go to file app\utils\chatgpt\chatgpt_cache_manager.py, change gpt_model=LLMModels._member_map_[stored_string["gpt_model"]] to gpt_model=LLMModels.gpt_4 or something.

This will override redis-stored-model. Now all chats will start with the model you defined, and changing model with /model will be maintained until you refresh the page.

    @classmethod
    async def read_context(cls, user_id: str, chat_room_id: str) -> UserGptContext:
        stored_string: dict[str, str | None] = {
            field: await cache.redis.get(key) for field, key in cls._get_string_fields(user_id, chat_room_id).items()
        }
        stored_list: dict[str, list | None] = {
            field: await cache.redis.lrange(key, 0, -1)
            for field, key in cls._get_list_fields(user_id, chat_room_id).items()
        }

        # if any of stored strings are None, create new context
        if any([value is None for value in stored_string.values()]):
            default: UserGptContext = UserGptContext.construct_default(
                user_id=user_id,
                chat_room_id=chat_room_id,
            )
            await cls.create_context(default)
            return default

        for field, value in stored_string.items():
            if value is not None:
                stored_string[field] = orjson_loads(value)
        for field, value in stored_list.items():
            if value is not None:
                stored_list[field] = [orjson_loads(v) for v in value]

        return UserGptContext(
            user_gpt_profile=UserGptProfile(**stored_string["user_gpt_profile"]),  # type: ignore
            gpt_model=LLMModels._member_map_[stored_string["gpt_model"]],  # type: ignore
            user_message_histories=[MessageHistory(**m) for m in stored_list["user_message_histories"]]
            if stored_list["user_message_histories"] is not None
            else [],
            gpt_message_histories=[MessageHistory(**m) for m in stored_list["gpt_message_histories"]]
            if stored_list["gpt_message_histories"] is not None
            else [],
            system_message_histories=[MessageHistory(**m) for m in stored_list["system_message_histories"]]
            if stored_list["system_message_histories"] is not None
            else [],
        )

[Torhamilton]

Yes this is clear. However pulling updates from repo, will over write entire file. .env is safest place to keep it.

def construct_default( cls, user_id: str, chat_room_id: str, gpt_model: LLMModels = LLMModels.gpt_4, // this will be revised by remote update )

[c0sogi]

ok I will add the feature on next commit.

[c0sogi]

aaea7faf81280678d5d2e88b38a97fede9d9fbf5

See the .env-sample file