-
Traceback (most recent call last):
File "/meta/cash/llm/InternImage/classification/main.py", line 661, in
main(config)
File "/meta/cash/llm/InternImage/classification/main.py", line 275, i…
-
**Describe the feature:**
The default GenAI response timeout appears to be around 60 seconds. This timeout should be adjustable per connector to account for varying models and responsiveness of the A…
-
### What happened?
Hello, llama.cpp experts! Thank you for creating such an amazing LLM Inference system. 😁
**However, while using this system, I encountered an unusual results when checking the spe…
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a sim…
-
https://github.com/nod-ai/sharktank/pull/39 started support for exporting a bs1-only variant of the llama model using a direct cache (instead of a paged cache) and was able to export prefill using bat…
-
Hi. Currently I'm trying to implement some large language models (LLM) with TorchSharp and got a nice demo ([here](https://github.com/K024/llm-sharp)). But when moving forward to more features I found…
K024 updated
11 months ago
-
The Error show as bellow
thread 'main' panicked at /Users/you/.cargo/registry/src/index.crates.io-6f17d22bba15001f/foreign-types-shared-0.3.1/src/lib.rs:72:9:
assertion failed: !ptr.is_null()
sta…
-
HW: MTL with ARC iGPU
OS: Ubuntu 22.04
Kernel: 6.5.0-41-generic
Ref: https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/install_linux_gpu.md
Problem1: cannot find GPU dr…
-
**Describe the bug**
When mounting a config file into kubernetes pod at /root/.memgpt/config, program exits because it cannot write to the config file.
```
ubuntu@terraform:~/workspace/home-ops-…
-
Hi,
after building vllm from source, the following error occures when running a multi-gpu inference using a local ray instance:
```
File "vllm/vllm/model_executor/layers/quantization/awq.py", lin…