search

mudler / LocalAI

:robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.
https://localai.io
MIT License
21.95k stars 1.68k forks source link

Add grpc service to query info about backend #2185

Open blob42 opened 2 months ago

blob42 commented 2 months ago

e Is your feature request related to a problem? Please describe.

Follow-up to #2086

I originally wanted to be able to query available models/speakers on the TTS backend and realized there were too many changes to keep this feature specific to TTS.

My first attempt was to add a gRPC service TTSInfoRequest to query the backend. I found out down the road that the backend grpc service is loaded with the model at the same time, however Info requests might not send any model infromation.

Describe the solution you'd like

A model could be loaded later using the same spawned service or tear-down and start a new one for the designated model. This would make it possible to add endpoints to query the backend for arbitrary information such as available models, params, speakers ... etc

I will start a draft PR.

Additional context

localai-bot commented 2 months ago

Hello! Thank you for submitting this feature request in our LocalAI repository. I'm a bot, an experiment of @mudler, auditing the tickets that get opened here.

From the details given in your issue, I understand that you'd like to make some changes to the back-end grpc service in order to add a new feature. You initially wanted to query available models and speakers on the TTS backend, but later realized that this feature should not be specific to TTS. Your proposed solution includes:

  1. Allowing the backends grpc service to be spawned without a model.
  2. Adding a service called Info() or Query to query arbitrary information from the backend.
  3. Adding an endpoint for TTS to list available models/speakers.

You mentioned that you'll be starting a draft PR for this. We appreciate the contribution and look forward to seeing the changes.

If you have any further questions or concerns, please don't hesitate to ask.

The ticket number for this issue is #2185, and the current label is [enhancement].