search

DevXT-LLC / ezlocalai

ezlocalai is an easy to set up local artificial intelligence server with OpenAI Style Endpoints.
MIT License
71 stars 13 forks source link

Documenting how to add more models #49

Open Utopiah opened 3 days ago

Utopiah commented 3 days ago

Hi, great project! I'm wondering how one can add more models. I'm thinking of e.g STT and TTS using CoquiAI or Whisper.cpp.

How do you recommend I do so rather than changing one of the current model by another?

Josh-XT commented 3 days ago

Screenshot_20240916-074637

Hello! Please expand the "Environment Variables" section in the readme. You'll want to use the hugging face URL for a GGUF model repository.

For example: https://huggingface.co/QuantFactory/Phi-3.5-mini-instruct-GGUF would just be QuantFactory/Phi-3.5-mini-instruct-GGUF

Utopiah commented 3 days ago

Thanks I'll try. Actually most of what you provide probably covers my need. I could enable all providers (as you linked before in https://josh-xt.github.io/AGiXT/2-Concepts/02-Providers.html ) to ezLocalai then whatever HF model available (maybe selected via https://llm.extractum.io for the available configuration, e.g under VRAM limit) if not the one you already selected.

Does it mean as-is ezlocalai is fully local and can work offline once properly setup?

isn't fully local and thus wouldn't work offline, due to relying on some remote API calls e.g Google (gTTS) as default for TTS? My bad seems gTTS isn't an API.

Also, and more out of curiosity, if I want to add another provider (I hope it's the right term) to do for example 3D model generation (using e.g TripoSR or shape-E) how would I do that correctly?