-
Hello,
I am working on trying Java projects but facing a serious problem. I found that many LLMs (except OpenAI GPTs), especially open-source ones (including Llama 70b, Code Llama, etc.), as well a…
-
**Description**
When a user performs a long-running inference request via HTTPServer, they may lose connection or intentionally abort the connection (ctrl-c from curl).
Ideally, the HTTP server will…
-
Following the steps here produces "Worker not connected" error when loading on port 3030.
Setup is a debian 12 LXC container on proxmox
Installed git
installed docker
followed the steps listed..…
-
a reference architecture (provided by the AI group)
a set of governance principles/framework (provided by the AI group)
-
[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question.
**Your Question**
How can i set config parameters for test data generati…
-
Because of the design of microchain (every llm response is just a function call):
- you can't use typical prompting techniques to improve responses ("think this through step-by-step", ...)
- you d…
-
Hi team,
I would like to use the LogitsPostProcessor in the [C++ Executor API](https://github.com/NVIDIA/TensorRT-LLM/blob/main/cpp/include/tensorrt_llm/executor/executor.h) to control the generatio…
-
Impressive work! The survey is very helpful for the community. I've asked my students to read it carefully 👍
I'd like to introduce our on-device LLM framework: [mllm](https://github.com/Ubiquitous…
-
**What would you like to be added/modified**:
This issue aims to build a cloud-edge collaborative inference framework for LLM on KubeEdge-Ianvs. Namely, it aims to help all cloud-edge LLM develop…
-
**Is your contribution request related to a problem? Please describe.**
I'm the maintainer of [OpenLLMetry](https://github.com/traceloop/openllmetry) where we instrument LLM providers for traces and …
nirga updated
1 month ago