-
### Bug Description
Want to leverage custom LLM and custom embedding to build basic RAG. custom LLM and embedding complete as expected when i am testing. But when i tried to use query() function, go…
-
**Is your feature request related to a problem? Please describe.**
I am composing a feature calling cloudflase model rest api by automaFetch as below. The rest api returns like 20 sec but automaFetch…
-
**The problem/use-case that the feature addresses**
Enable use of Valkey with LLM applications for semantic LLM caching, semantic conversation cache, LLM semantic routing
**Description of the fe…
-
> I'm going to test this with a fresh install of `llm` to make sure it doesn't break.
>
> Although... here's an interesting callenge with LLM: I frequently run many different copies of it against th…
-
https://docs.anthropic.com/en/docs/build-with-claude/pdf-support
> The new Claude 3.5 Sonnet (`claude-3-5-sonnet-20241022`) model now supports PDF input and understands both text and visual content…
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a sim…
-
**Bug Description**
Use OpenAI compatible API to connect local LLM which created by GPUStack, but when sending a message, it returns this error:
```
Unexpected token 'D', "[DONE] " is not valid JSO…
-
## Description
Mlflow [introduced inference parameters](https://mlflow.org/docs/latest/models.html#inference-params) in 2.6.0.
## Context
Passing parameters at inference time to avoid retraining …
-
### System Info / 系統信息
- vllm 0.5.3
- transformer 4.44.0
- torch 2.3.1
### Who can help? / 谁可以帮助到您？
@sixsixcoder @zr
### Information / 问题信息
- [X] The official example scripts / 官方的示例脚…
-
OS type
Ubuntu
Description
When running the example Translation using Docker Compose, one of the images takes additional time to pull a model from the Huggingface upon startup. During this period…