-
A Triton inference server might be useful for the open-source models
https://github.com/triton-inference-server
-
[LM Studio support](https://github.com/langchain-ai/opengpts/issues/9) and [OpenRouter support](https://openrouter.ai/docs#models)
- [ ] Refactor base data structures
-
### Which API Provider are you using?
OpenRouter
### Which Model are you using?
Claude 3.5 Sonnet
### What happened?
Unable to reference URL's on idx.dev enviroment. Following error is thrown:
…
-
I'm trying to use browser-use with vLLM as the backend LLM service, but I'm running into issues. Has anyone successfully integrated vLLM with this package?
## Current Setup
I'm using the followi…
-
Once the text generation finishes, there actually appears to be no way to continue generating, the submit button is greyed out and clicking it just errors out. I am using OpenAI endpoint in Koboldcpp …
-
Due to the fact, that OpenAI chose to not accept prepaid-creditcards anymore. (This is a microsoft-thing...)
I cannot use the OpenAI-API anymore.
Langchain is automatically used to use the Embeddi…
-
LiteLLM, Llama.cpp, Vllm, OpenRouter, TogetherAI,....
Everyone has an OpenAI-like api nowadays. It would be really great to be able to use them in place of OpenAI api.
Thanks!
-
tested with the default address and port
http://127.0.0.1:11434/
llama3.1
llama3.1:latest
mimicking the oobabooga setup did not result in a connection either. https://docs.sillytavern.app…
-
### Which API Provider are you using?
OpenRouter
### Which Model are you using?
Sonnet
### What happened?
…
-
### Describe the feature
It'd be awesome if we could add support for Groq Cloud. It's fast and has some excellent new models, including Llama 3.1 405B(https://groq.link/llama3405bblog / https://con…