Zipstack / unstract

No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents
https://unstract.com
GNU Affero General Public License v3.0
354 stars 30 forks source link

FEAT: Persist Token Usage #378

Closed tahierhussain closed 1 month ago

tahierhussain commented 1 month ago

What

Backend:

  1. Split the logic for retrieving aggregated token usage into a separate method to facilitate reuse when returning prompt output data.
  2. Updated the prompt output serializer's to_representation method to include token usage information.

Frontend:

  1. Updated TokenUsage component to fetch and display token usage details based on tokenUsageId.
  2. Implemented global state to persist token usage data.

-

Why

The token usage data was not persisted earlier.

-

How

The run_id was already available in the prompt output manager table. Using that run_id, the aggregated token usage information is retrieved and included in the response.

-

Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)

Yes, there is a possibility of affecting existing features since the prompt output manager has been modified to accommodate the token usage persistence changes.

-

Database Migrations

NA

Env Config

NA

Relevant Docs

NA

Related Issues or PRs

NA

Dependencies Versions

NA

Notes on Testing

NA

Screenshots

Multi Pass: Screenshot from 2024-06-02 21-25-03

Coverage Modal: Screenshot from 2024-06-02 21-25-10

Checklist

I have read and understood the [Contribution Guidelines]().

sonarcloud[bot] commented 1 month ago

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud