search

mudler / LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed inference
https://localai.io
MIT License
22.73k stars 1.73k forks source link

wired and missing api calls in /metrics #2207

Open ck3d opened 3 months ago

ck3d commented 3 months ago

I created multiple LocalAI integrations tests with the NixOS test infrastructure. At end of each test, I dump the Prometheus metrics file as JSON format. The following test https://github.com/ck3d/nix-local-ai/blob/main/local-ai/tests.nix#L181 creates following metrics.json.

The test calls following endpoinds each once: /readyz /v1/models /v1/chat/completions /v1/edits /v1/completions /metrics

The metrics files show wired api calls:

  1. method "GETT" which should be a "GET"
  2. endpoind "/metricsls", "/v1/compl", "/v1/completionstions" was not called
  3. no posts are listet

I did not check if the released LocalAI 2.13.0 has the same issue.

The test can be executed on a machine, which has nix installed, with following command: nix build github:ck3d/nix-local-ai/1d0ae56380824349fd0590a2e1352ea42b129727#local-ai.tests.llama

localai-bot commented 3 months ago

Hello,

Thank you for bringing this to our attention. As a bot, an experiment of @mudler, I have reviewed the details you've provided in the Github issue #2207. It seems that there are discrepancies in the API calls and their corresponding methods in the generated Prometheus metrics file. Moreover, some endpoints were not called at all.

The test output you shared indicates that some of the API calls are missing the correct HTTP method or have incorrect endpoint names. We will need to investigate this further and address the issue accordingly.

To expedite the resolution process, it would be helpful if you could confirm if the same issue persists in the released version of LocalAI 2.13.0. If it does, please provide any additional logs, screenshots, or test results that can assist in identifying the root cause.

For others experiencing the same issue, I recommend checking the status of this ticket and following the updates for any resolution. If necessary, feel free to comment on the issue and provide your own insights or tests results to help improve the investigation.

In the meantime, please let us know if there are any other concerns or issues you're facing with the LocalAI project. We appreciate your help in making LocalAI better and more efficient.

ck3d commented 1 month ago

Issue still present for version 2.18.1