tsterbak / promptmage

simplifies the process of creating and managing LLM workflows.
https://promptmage.io
MIT License
67 stars 4 forks source link

[BUG] Prompt Version not Updating in Side Panel #7

Closed LuciAkirami closed 2 weeks ago

LuciAkirami commented 2 weeks ago

Here are the results from the database when I changed the prompt (user) three times

Initial id name system user version template_vars active
405263dc-26bb-4f09-9960-5041760edb6d qa_bot You are an helpful AI assistant Answer the following user question:{query} 1 query 0
After updating once id name system user version template_vars active
405263dc-26bb-4f09-9960-5041760edb6d qa_bot You are an helpful AI assistant Answer the following human question:{query} 1 query 0
After updating twice id name system user version template_vars active
405263dc-26bb-4f09-9960-5041760edb6d qa_bot You are an helpful AI assistant Answer the following humanoid question:{query} 1 query 0

I have checked the sqlite_backend.py's update_prompt() function of SQLiteBackend class. Here's the code

storage/sqlite_backend.py - line 116

    def update_prompt(self, prompt: Prompt):
        """Update an existing prompt in the database by id.

        Args:
            prompt (Prompt): The prompt to update.
        """
        session = self.Session()
        try:
            existing_prompt = session.execute(
                select(PromptModel).where(PromptModel.id == prompt.id)
            ).scalar_one_or_none()

            if not existing_prompt:
                raise PromptNotFoundException(
                    f"Prompt with name {prompt.name} not found."
                )

            existing_prompt.system = prompt.system
            existing_prompt.user = prompt.user
            existing_prompt.active = prompt.active

            session.commit()

Here, we are updating from the previous system and user prompt to the new system and user prompt. But nowhere we are increasing the version number by 1 (which was present in the previous code)

Here's the solution (just add a single line)

    def update_prompt(self, prompt: Prompt):
        """Update an existing prompt in the database by id.

        Args:
            prompt (Prompt): The prompt to update.
        """
        session = self.Session()
        try:
            existing_prompt = session.execute(
                select(PromptModel).where(PromptModel.id == prompt.id)
            ).scalar_one_or_none()

            if not existing_prompt:
                raise PromptNotFoundException(
                    f"Prompt with name {prompt.name} not found."
                )

            existing_prompt.system = prompt.system
            existing_prompt.user = prompt.user
            existing_prompt.active = prompt.active
            # updating the version
            existing_prompt.version += 1
            # updating the version

            session.commit()

After doing this, here's the result from the database when a prompt update is performed in the UI

Initial id name system user version template_vars active
ae49eebd-e299-4f16-81f3-366fcb9c91ae qa_bot You are an helpful AI assistant Answer the following user question:{query} 1 query 0
After updating once id name system user version template_vars active
ae49eebd-e299-4f16-81f3-366fcb9c91ae qa_bot You are an helpful AI assistant Answer the following human question:{query} 2 query 0

Opening a PR for the same

LuciAkirami commented 2 weeks ago

I have noticed another bug. This is when let's say, I click on the Edit Promptand do not edit anything and click on Save Prompt. Then even though the user prompt and system prompt is the same, the version gets updated. For this I made changes to the code as follows

Existing Code: storage/sqlite_backend.py - line 116

    def update_prompt(self, prompt: Prompt):
        """Update an existing prompt in the database by id.

        Args:
            prompt (Prompt): The prompt to update.
        """
        session = self.Session()
        try:
            existing_prompt = session.execute(
                select(PromptModel).where(PromptModel.id == prompt.id)
            ).scalar_one_or_none()

            if not existing_prompt:
                raise PromptNotFoundException(
                    f"Prompt with name {prompt.name} not found."
                )

            existing_prompt.system = prompt.system
            existing_prompt.user = prompt.user
            existing_prompt.active = prompt.active

            session.commit()

Update I suggested above (to sovlve the "[BUG] Prompt Version not Updating in DB and UI"):

    def update_prompt(self, prompt: Prompt):
        """Update an existing prompt in the database by id.

        Args:
            prompt (Prompt): The prompt to update.
        """
        session = self.Session()
        try:
            existing_prompt = session.execute(
                select(PromptModel).where(PromptModel.id == prompt.id)
            ).scalar_one_or_none()

            if not existing_prompt:
                raise PromptNotFoundException(
                    f"Prompt with name {prompt.name} not found."
                )

            existing_prompt.system = prompt.system
            existing_prompt.user = prompt.user
            existing_prompt.active = prompt.active
            # updating the version
            existing_prompt.version += 1
            # updating the version

            session.commit()

Updating it again to fix the new bug (that is do not change the version if no changes are made to the user and system prompt)

    def update_prompt(self, prompt: Prompt):
        """Update an existing prompt in the database by id.

        Args:
            prompt (Prompt): The prompt to update.
        """
        session = self.Session()
        try:
            existing_prompt = session.execute(
                select(PromptModel).where(PromptModel.id == prompt.id)
            ).scalar_one_or_none()

            if not existing_prompt:
                raise PromptNotFoundException(
                    f"Prompt with name {prompt.name} not found."
                )

            # Code Change
            # If changes are made to user prompt and system prompt
            if (existing_prompt.system != prompt.system or existing_prompt.user != prompt.user):
                existing_prompt.system = prompt.system
                existing_prompt.user = prompt.user
                existing_prompt.active = prompt.active
                existing_prompt.version += 1
            # If no changes made to user prompt and system prompt
            else:
                # This case happens when we click on the Activate Prompt, which inturn calls the update_prompt
                existing_prompt.active = prompt.active
            # Code Change

            session.commit()

So when we click on the Activate Prompt in the UI, it inturn calls the update_prompt of the SQLiteBackend. As there are no changes made in the user prompt and system prompt, we just update the active variable

tsterbak commented 2 weeks ago

Fixed with the latest release 0.1.2.