-
Can the model be loaded only once instead of waiting for the load to complete each time?
-
Follow [agent comp](https://github.com/opea-project/GenAIComps/tree/main/comps/agent/langchain) started the TGI and Agent service, agent service failed.
[start agent microservices with tgi](
https…
-
希望发布0.7.1版本的同时，能够提供相应docker镜像，像开源推理框架vllm，每发布一个版本都会同时发布一个相应版本的docker镜像，这样可以避免了复杂且重复的依赖环境安装问题。
-
Does vlmeval support multi card inference and batch size > 1?
-
- [ ] Fp8 kv-cache
- [ ] Kv-cache prefix reuse
- [ ] Grammar constrained speedup
- [ ] `torch.compile` like speedups
- [ ] Simple one-liner `pip install`
- [ ] Multi lora support (lorax kind of)
…
-
### System Info
`
text-generation-launcher 2.1.0
`
### Information
- [X] Docker
- [X] The CLI directly
### Tasks
- [ ] An officially supported command
- [ ] My own modifications
### Reprod…
-
I've been serving `codellama/CodeLlama-7b-hf` using openshift AI `Caikit TGIS ServingRuntime for KServe` and trying to interact with it using langchain via `caikit-nlp-client` and [caikit_tgis_langcha…
-
There's a few areas where the initial experience of using Outlines is a little clunky.
## Installing Outlines
For one, none of the inference backends [are dependencies](https://github.com/dottx…
-
### System Info
```
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4…
-
1. 基于单一的category画像、基于多种用户画像的交集（白盒圈人）
1. 依赖运营经验，根据tag、cate画像，或者点击行为，圈定用户
1. 有的下发量极大，未考虑用户是否对item感兴趣
1. 人工选tag不精准，未能把item高潜用户圈进来（如明星漏税只圈了财经）
1. why基于lookalike定向找到相似用户
1. 完全满足人群定向的条件…