-
There are cases where it would be nice for the LLM to have the context of another file I have open. I don't have a good idea yet on the implementation specifics but I'm wondering if you would be open …
-
TEI_EMBEDDING_ENDPOINT
TEI_RERANKING_ENDPOINT
TGI_LLM_ENDPOINT
```yaml
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
apiVersion: gmc.opea.io/v1alpha3
kind: GM…
-
### Motivation.
There have been quite a lot requests for releasing the resources taken by the `LLM` class, e.g. in https://github.com/vllm-project/vllm/issues/3874 and https://github.com/vllm-proje…
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain.js documentation with the integrated search.
- [X] I used the GitHub search to find a …
-
### The Feature
The `/health` endpoint provides a lot of detailed information in the `healthy_endpoints`/`unhealthy_endpoints` properties. We usually expose health endpoints without authentication …
-
**Describe the package you'd like added**
`vllm` has become a popular inference server for LLMs: https://github.com/vllm-project/vllm
**Describe how this package fits in with the project**
GenAI/…
-
I will list the test results of various open-source models here. You can refer to these data to select models and configure devices. Of course, the evaluation of LLM is quite subjective. I also sugges…
-
As getting OpenAI key is pain because it expires every 3 months and getting a new one needs a new phone number for verification. So adding option for an open source LLM will be great. Allow user to ch…
-
## [LangChain Development](https://app.pluralsight.com/library/courses/langchain-development/table-of-contents)
by [Tom Taulli](https://app.pluralsight.com/profile/author/tom-taulli)
founder : H…
-
### System Info
CPU: x86_64
GPU: L40s
TensorRT-LLM commit: 118b3d7e7bab720d8ea9cd95338da60f7512c93a
cuda: 12.3
TRT version: 9.3.0.1
### Who can help?
@byshiue
### Information
- [X] The o…