letta-ai / letta

Letta (formerly MemGPT) is a framework for creating LLM services with memory.
https://letta.com
Apache License 2.0
13.13k stars 1.44k forks source link

Dev Portal - Unable to create tool #1631

Open raolak opened 3 months ago

raolak commented 3 months ago

Describe the bug I tried to create tool with default function populated.. The post request throws this error

{ "detail": [ { "type": "missing", "loc": [ "body", "json_schema" ], "msg": "Field required", "input": { "name": "astra_test", "source_code": "import random\n\ndef roll_d20(self) -> str:\n \"\"\"\n Simulate the roll of a 20-sided die (d20).\n\n This function generates a random integer between 1 and 20, inclusive,\n which represents the outcome of a single roll of a d20.\n\n Returns:\n int: A random integer between 1 and 20, representing the die roll.\n\n Example:\n >>> roll_d20()\n 15 # This is an example output and may vary each time the function is called.\n \"\"\"\n dice_role_outcome = random.randint(1, 20)\n output_string = f\"You rolled a {dice_role_outcome}\"\n return output_string" }, "url": "https://errors.pydantic.dev/2.7/v/missing" } ] }

raolak commented 3 months ago

From UI call payload need to include source_type and json_chema = {name}

{ "name": "test_tool", "source_type": "python", "json_schema": {"name":"test_tool"}, "source_code": "import random\n\ndef roll_d20(self) -> str:\n \"\"\"\n Simulate the roll of a 20-sided die (d20).\n\n This function generates a random integer between 1 and 20, inclusive,\n which represents the outcome of a single roll of a d20.\n\n Returns:\n int: A random integer between 1 and 20, representing the die roll.\n\n Example:\n >>> roll_d20()\n 15 # This is an example output and may vary each time the function is called.\n \"\"\"\n dice_role_outcome = random.randint(1, 20)\n output_string = f\"You rolled a {dice_role_outcome}\"\n return output_string" }

raolak commented 3 months ago

Is the pre requisiet for create tools api is to generate jason_schema?

@router.post("/tools", tags=["tools"], response_model=ToolModel) async def create_tool( request: CreateToolRequest = Body(...), ): """ Create a new tool """ try: return server.create_tool( json_schema=request.json_schema, source_code=request.source_code, source_type=request.source_type, tags=request.tags ) except Exception as e: raise HTTPException(status_code=500, detail=f"Failed to create tool: {e}")

return router

Here is the suggestion

At present beacuse tool got created without json schema and tags...the dev portal tools ui is broken

Nostoi commented 3 months ago

Check the URL of the API call.

I'm working from memory, but I was testing the API with CURL requests and comparing it to the dev portal it seems as - at least in some cases - the name of the tool is being appended to /admin/tools rather than being embedded in the JSON object.

For example, a tool named "great-tool" is being added the URL and the portal is making a request at /admin/tool/create-tool rather than /admin/tool.

As a result, you may be seeing 504 (?) errors in the logs or some other error when you attempt to create from the dev portal.

This seems to be a bug that needs to be fixed. Again, I'm working from memory but that's what I found.

raolak commented 3 months ago

Thanks Nostoi for the update

I am copying my networlk trace from dev tool again. I had to create tool using client...dev portal i think implementation is incomplete

POST - http://:/api/tools

Payload: {"name":"atrsa_dice","source_code":"import random\n\ndef roll_d20(self) -> str:\n \"\"\"\n Simulate the roll of a 20-sided die (d20).\n\n This function generates a random integer between 1 and 20, inclusive,\n which represents the outcome of a single roll of a d20.\n\n Returns:\n int: A random integer between 1 and 20, representing the die roll.\n\n Example:\n >>> roll_d20()\n 15 # This is an example output and may vary each time the function is called.\n \"\"\"\n dice_role_outcome = random.randint(1, 20)\n output_string = f\"You rolled a {dice_role_outcome}\"\n return output_string"}

Status 422 Unprocessable Entity -

Make json_schema optional, as dev tool can not pass the jsnon_schema. It need to be generated on api side class CreateToolRequest(BaseModel): json_schema: dict = Field(..., description="JSON schema of the tool.") source_code: str = Field(..., description="The source code of the function.") source_type: Optional[Literal["python"]] = Field(None, description="The type of the source code.") tags: Optional[List[str]] = Field(None, description="Metadata tags.") update: Optional[bool] = Field(False, description="Update the tool if it already exists.")

Response { "detail": [ { "type": "missing", "loc": [ "body", "json_schema" ], "msg": "Field required", "input": { "name": "atrsa_dice", "source_code": "import random\n\ndef roll_d20(self) -> str:\n \"\"\"\n Simulate the roll of a 20-sided die (d20).\n\n This function generates a random integer between 1 and 20, inclusive,\n which represents the outcome of a single roll of a d20.\n\n Returns:\n int: A random integer between 1 and 20, representing the die roll.\n\n Example:\n >>> roll_d20()\n 15 # This is an example output and may vary each time the function is called.\n \"\"\"\n dice_role_outcome = random.randint(1, 20)\n output_string = f\"You rolled a {dice_role_outcome}\"\n return output_string" }, "url": "https://errors.pydantic.dev/2.7/v/missing" } ] }

The current create tools api implementation, the server.create_tool function doesn't calls json_schema = generate_schema(func, func_name) before saving the tool record in to db. We need to generate the schema and set the json_schema field and then save the record

May be add this support inside api create_tool function

Example

def get_tool_function(source_code: str):
    """
    Processes the provided source code, creates a function object, and generates a JSON schema.

    Args:
        source_code (str): The source code containing the function definition.

    Returns:
        tuple: A tuple containing the generated JSON schema and the source type.

    Raises:
        HTTPException: If processing the source code fails.
    """
    try:
        # Create a local context to hold the function object
        local_context = {}

        # Execute the source code to create the function object
        exec(source_code, globals(), local_context)

        # Retrieve the function object (assuming there's only one function in the code)
        func_name = next(iter(local_context))
        func = local_context[func_name]

        # Generate the JSON schema from the function object
        json_schema = generate_schema(func, func_name)

        # Set the source type to 'python'
        source_type = "python"

        return json_schema, source_type

    except Exception as e:
        raise HTTPException(
            status_code=400, detail=f"Failed to process source code: {e}"
        )

# Usage in the main function
@router.post("/tools", tags=["tools"], response_model=ToolModel)
async def create_tool(
    request: CreateToolRequest = Body(...),
    user_id: uuid.UUID = Depends(get_current_user_with_server),
):
    """
    Create a new tool
    """

    # Initialize json_schema and source_type
    json_schema = request.json_schema
    source_type = request.source_type

    # If json_schema is not provided, generate it from the source code
    if not json_schema and request.source_code:
        json_schema, source_type = get_tool_function(request.source_code)

    try:
        return server.create_tool(
            json_schema=json_schema,
            source_code=request.source_code,
            source_type=source_type,
            tags=request.tags,
            user_id=user_id,
            exists_ok=request.update,
        )
    except Exception as e:
        print(e)
        raise HTTPException(
            status_code=500, detail=f"Failed to create tool: {e}, exists_ok={request.update}"
        )

    return router