Statistics page in WebUI and through an API

mudler / LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed inference

https://localai.io

MIT License

23.24k stars 1.76k forks source link

Statistics page in WebUI and through an API #2763

Open SuperPat45 opened 2 months ago

SuperPat45 commented 2 months ago

I would love to have a full statistics page in the WebUI showing the current activity of the server (by seconds):

number of call for each endpoint
number of failed request
number token IN/OUT
CPU load
RAM usage
GPU load
VRAM usage
...

Also, these data can be represented in charts by days like the OpenAI usage page: https://platform.openai.com/usage/activity

These data could also be available to a new /stats API endpoint.

mudler commented 2 months ago

I'd love to have this in too :+1:

dave-gray101 commented 2 months ago

We definitely need to further enhance what we're tracking, but the infrastructure we already have is open telemetry based - so before we spend time working on our own UI, we should probably run some searches for permissively licensed otel compatible display code. At the end of the day something like grafana is always going to be prettier than what we integrate, so I'm assuming that this page should be kept pretty simple