Feature Request: Add Sambanova and Huggingface Inference API variables

glibsonoran / Plush-for-ComfyUI

Custom node for ComfyUI/Stable Diffustion

GNU General Public License v3.0

143 stars 12 forks source link

Feature Request: Add Sambanova and Huggingface Inference API variables #115

Open EnragedAntelope opened 4 hours ago

EnragedAntelope commented 4 hours ago

Hi, I love Plush nodes and use them daily! Is it possible to include Sambanova? It uses standard OpenAI chat completion (https://api.sambanova.ai/v1/chat/completions) , it just needs the API key set and a model selected. It is free and offers Meta Llama 405B! https://cloud.sambanova.ai/apis

Separately, Huggingface offers free API inference, not sure if that is something that could also be included? https://huggingface.co/docs/api-inference/index

Thank you!

glibsonoran commented 3 hours ago

I don't have an account with Sambanova so it won't let me see anything using that link. However if it is truly OpenAI compliant you should be able to use: "Local app (URL)" or "OpenAI compatible http POST (URL)" on the AI_Services menu if you first put your API key into an LLM_KEY environment variable. Then just add the URL for the service in the LLM_URL field and it should work.

glibsonoran commented 3 hours ago

Note that using "OpenAI compatible http POST (URL)" will require a URL that has a [your http address]/v1/chat/completions path as its format. "Local app (URL)" will automatically format the path for you, so you can just leave the longer path in place.