otto8-ai / otto8

Open source AI Agent Platform
Apache License 2.0
8 stars 9 forks source link

Not able to save images. #514

Open sangee2004 opened 4 days ago

sangee2004 commented 4 days ago

Steps to reproduce the problem:

  1. Create an agent with Images tool bundle and workspace Files
  2. Chat with the agent and ask following:
  3. generate an image of cow
  4. Save this image in file cow.png

This results in image being saved with content - "/api/threads/t18t2m9/file/generated_image_324b2a32.png"

          {
            "content": "ave this image in file cow.png",
            "role": "user"
          },
          {
            "content": "",
            "role": "assistant",
            "tool_calls": [
              {
                "function": {
                  "arguments": "{\"filename\":\"cow.png\",\"content\":\"/api/threads/t18t2m9/file/generated_image_324b2a32.png\"}",
                  "name": "workspaceWrite"
                },
                "id": "call_ejo80bgZTEJD44hDhSCPTAVQ",
                "type": "function"
              }
            ]
          },
          {
            "content": "Wrote 54 bytes\n",
            "name": "workspaceWrite",
            "role": "tool",
            "tool_call_id": "call_ejo80bgZTEJD44hDhSCPTAVQ"
          }

Debug logs - calls - 2024-11-10T121811.761.json

Expected Behavior: We should be able to save the binary contents of the image.

Note - This causes problem when other tools like vision wants to access this file like - https://github.com/otto8-ai/otto8/issues/431#issuecomment-2466826619

cjellick commented 4 days ago

to be clear, the image is saved to the workspace upon generation. sangeetha is effectively trying to copy the image. it is just too big for the workspace read or write files to work with. The llm essentially decided to do the wrong thing and save the reference to the image file, not the image itself.