-
This is not an error specific to this project but I have been unable to get the repos on `llama_index` to work on my computer. I have looked around on the llama_index issue forums and the only advice…
-
llama.cpp now supports grammars:
https://til.simonwillison.net/llms/llama-cpp-python-grammars
Is that something that will come to candle?
It sounds like the approach taken in this python libr…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [ ] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
First of all, when pp_size = 1, everything is good with tp_size = 1,2,4,8.
My test on pipeline parallelism (pp_size > 1) always failed with different error in the last few rows in this post.
Firs…
-
### System Info
GPU 2* A30, TRT-LLM branch main, commid id: 66ef1df492f7bc9c8eeb01d7e14db01838e3f0bd
### Who can help?
_No response_
### Information
- [x] The official example scripts
- [ ] …
-
### 起始日期 | Start Date
01062024
### 实现PR | Implementation PR
Qwen-VL 有加速框架吗？如vLLm 或者 llama.cpp等类似框架， 我觉得推理速度有点慢，一张图片我的3090 需要3-5秒
### 相关Issues | Reference Issues
5️⃣
### 摘要 | Summary
Qwen-VL 有加…
-
Open to suggestions / assistance on how to make installation easier and less error prone.
One thought is to add better platform detection to the cmakelists and provide better docs / links if requir…
-
lava-cli.dir\linkLibs.rsp
C:\w64devkit\bin/ld.exe: C:/w64devkit/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/lib/../lib/libpthread.a(libwinpthread_la-thread.o):thread…
-
Hi,
The codeshell model: https://huggingface.co/WisdomShell/CodeShell-7B-Chat-int4
Since CodeShell is already supported by llama.cpp, please help add CodeShell to ollama/library. Thanks!