meta-llama / llama-stack-client-python

Python SDK for Llama Stack
Apache License 2.0
77 stars 19 forks source link

print the error sent from the server #44

Closed chuenlok closed 6 days ago

chuenlok commented 1 week ago

What does this PR do?

The assert errors from the server are not returned to the client and hid by the 500 internal server error. Updated the server code (https://github.com/meta-llama/llama-stack/pull/493) to send the error and this client can print the error.

Test Plan

Please describe:

  1. Kick off the server with llama stack run /Users/henrytai/.llama/distributions/llamastack-ollama/ollama-run.yaml --port 5050
  2. Trigger an error case from client. The example is PYTHONPATH=. python -m examples.agents.client localhost 5050 from the llama-stack-apps repo. The issue details: "This is a known issue with Llama3.2-3B-Instruct on ollama, where the model produces bad tool calling outputs. In this case, the server agent fails due to assert error that "code_interpreter" is not in the AgentConfig's tools field" but the code_interpreter is not really necessary as it is not needed in Llama3.1-8B.

client log:

inference> assistant<|end_header_id|>

Switzerland is a beautiful country with a rich history, stunning mountains, and vibrant cities. Here are three must-visit places in Switzerland:

1. **Geneva**: Geneva is a global hub for international diplomacy and is often referred to as the "City of Peace". It's home to ...

These are just a few examples of Switzerland's amazing attractions!
500: Internal server error: An unexpected error occurred. Tool code_interpreter not found
inference> brave_search.call(query="what is special about #1")...

Showing the real error in the client: Tool code_interpreter not found

Sources

Please link relevant resources if necessary.

Before submitting

chuenlok commented 1 week ago

@yanxi0830 Can you help review this?

yanxi0830 commented 1 week ago

Thanks! Left some questions. Could you rebase the change on top of current main? There have been some refactors done on the agents lib.

chuenlok commented 1 week ago

Thanks! Left some questions. Could you rebase the change on top of current main? There have been some refactors done on the agents lib.

Rebased!