search

smallcloudai / refact-lsp

LSP server for Refact, suitable for Sublime Text, and other editors
BSD 3-Clause "New" or "Revised" License
24 stars 12 forks source link

Metering #244

Closed valaises closed 1 month ago

valaises commented 1 month ago

Implemented session-wise metering. To make it work with Refact Self-Hosted, these changes are required: https://github.com/smallcloudai/refact/pull/448

To check metering stats, use handler v1/metering if you made some chat requests, you will see something like this:

[
  {
    "model": "gpt-4o",
    "pp1000t_prompt": 5000,
    "pp1000t_generated": 15000,
    "metering_prompt_tokens_n": 275,
    "metering_generated_tokens_n": 48,
    "requests_cnt": 2
  },
  {
    "model": "gpt-4-0125-preview",
    "pp1000t_prompt": 10000,
    "pp1000t_generated": 30000,
    "metering_prompt_tokens_n": 268,
    "metering_generated_tokens_n": 45,
    "requests_cnt": 2
  }
]