-
Hi, having this issue with connecting to external llms.
Enviroment server for remote LLM:
- Amd 79503xd
- 64 GB RAM
- 2x 7900xtx
- Using LM-STUDIO fosr hosting LLM server
Enviroment Cli…
-
### System Info
x86_64
755G
nvidia T4
ubuntu 22.04
trtllm version : https://github.com/NVIDIA/TensorRT-LLM/archive/9691e12bce7ae1c126c435a049eb516eb119486c.zip
pip install tensorrt-llm==0.11…
-
### What happened?
I am using the llama-2-7b-chat.Q4_K_M.gguf and trying to run it using llama-cpp
but I am not getting the actual output .I am getting output as # , not as any string.
### Nam…
-
Update: I used to run ollama on this chromebook when tinyllama came out and it ran great.
### What is the issue?
![image](https://github.com/ollama/ollama/assets/13264408/e37d1a70-8d92-4281-88…
-
### System Info
4*NVIDIA L20
### Who can help?
_No response_
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially suppor…
-
Hi,
I am unable to import LlamaCpp in IPEX
CODE : from ipex_llm.langchain.llms import LlamaCpp
ERROR
Cell In[5], [line 1](vscode-notebook-cell:?execution_count=5&line=1)
----> [1](vscode-note…
-
When "pip install ipex-llm[cpp]", then "init-ollama.bat", it runs on CPU:
" ... msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="31.6 GiB" ... "
But when "pip install …
-
### System Info
-CPU architecture: amd64
-Operating System: Windows 11
-Python version: 3.11.5
-TensorRT-LLM version: 0.10.0
-CUDA version: 12.5
-torch version: 2.2.0+cu121
### Who can help?
_…
-
### System Info
- CPU architecture : x86_64
- CPU/Host memory size : 32 GB
- GPU name L4 at g2-standard-8 (GCP)
- GPU memory size 24GB
- TensorRT-LLM branch or tag (e.g., main, v0.10.0)
- Nvi…
-
### Your current environment
```text
GPU 0: NVIDIA H100 80GB HBM3
GPU 1: NVIDIA H100 80GB HBM3
GPU 2: NVIDIA H100 80GB HBM3
GPU 3: NVIDIA H100 80GB HBM3
GPU 4: NVIDIA H100 80GB HBM3
GPU 5: NV…