search

All-Hands-AI / OpenHands

🙌 OpenHands: Code Less, Make More
https://all-hands.dev
MIT License
37.36k stars 4.23k forks source link

Display API costs in frontend #5257

Open neubig opened 4 hours ago

neubig commented 4 hours ago

What problem or use case are you trying to solve?

Direct communication from a user:

the unknown API costs are a fairly intimidating part of the current user experience. I tried running a relatively simple query and wasn’t sure if it was going to cost 1 cent or 10 dollars (it ended up costing 10 cents on GPT4o). is there any way to give users an upfront ballpark of costs (even order of magnitude)? could there be an interactive mode where I can press yes / no on exact costs associated w/ individual steps? (e.g., “Keep going? The next step will cost $0.02 cents”)

Describe the UX of the solution you'd like

It'd be good to give an idea of (1) how much the whole agent trajectory cost, and (2) how much the past few steps cost.

Do you have thoughts on the technical implementation?

  1. Through the backend API, we will have to expose the costs of each step and the total cost so far.
  2. Somewhere in the frontend we can have a user click on a button to open up statistics about cost (and possibly other things).
github-actions[bot] commented 4 hours ago

OpenHands started fixing the issue! You can monitor the progress here.

github-actions[bot] commented 4 hours ago

An attempt was made to automatically fix this issue, but it was unsuccessful. A branch named 'openhands-fix-issue-5257' has been created with the attempted changes. You can view the branch here. Manual intervention may be required.

neubig commented 4 hours ago

Based on a follow-up discussion with @rbren , we should maybe have statistics put in a separate window to prevent clutter in the main interface. But the window could be opened with a button.