-
### What is the issue?
It's again the #4334 issue. I have multiple local CPU nodes and using ollama behind the litellm proxy.
The issue is with embedding call for **snowflake-arctic-embed** model.…
-
### Discussed in https://github.com/karthink/gptel/discussions/169
Originally posted by **eastbowl** January 4, 2024
Hi,
This is probably a very naive question (I'm a novice and non-progra…
-
# Prerequisites
Version 0.2.84 or 0.2.85 and using create_chat_completion method.
Tried different GGUF models.
Please answer the following questions for yourself before submitting an issue.
…
-
**Documentation description**
Just a relevant example of how to use with node-llama-cpp
Steps include installing node-llama-cpp
installing langchain/community
and here's relevant code:
```
impor…
-
I'm currently trying out the ollama app on my iMac (i7/Vega64) and I can't seem to get it to use my GPU.
I have tried running it with num_gpu 1 but that generated the warnings below.
`
2023/11/…
-
Hello 作者你好, 我正在复现 PowerInfer, 对比 llama.cpp 和 PowerInfer 的性能。在基准测试阶段我遇到了一些意料之外的结果。
# 环境
- 代码
- PowerInfer: commit: 2217e7fb8c2409f2d4927c5fe15fd6c9ef289e96
- llama.cpp: commit: 6bb4908a1715…
-
### Description
I decided to try the popular configuration min_p = 0.1 and temp = 1.5 or higher.
I get the following **result**:
![image](https://github.com/user-attachments/assets/faeaae78-c3d…
-
### Issue Description
I am currently running the **Exo** framework on my Windows device. Initially, I encountered compatibility errors because the `main.py` file was only supported on macOS. Howeve…
-
I followed the readme, but i can't get llama-cpp to run on my 4090.
```
set CMAKE_ARGS=-DLLAMA_CUBLAS=on
set FORCE_CMAKE=1
pip install llama-cpp-python --upgrade --force-reinstall --no-cache-di…
-
Nothing should actually prevent a windows build. It's more matter of using the right compiler, dependencies, and setting up CI. MSVC does not build out of the box. This is likely an issue with `drama_…