-
**Describe the package you'd like added**
`llama.cpp` has become a popular inference server for LLMs. Additionally, `llama-cpp-python` is commonly used to connect from Python to `llama.cpp`.
- `l…
-
对于官方发布的 LLMS 大模型,建议在未来可以附上 awq 和 gptq 的量化版本。这种做法几乎没有成本,但却能帮助许多缺乏 GPU 的潜在用户。这会让用户在使用模型时更加方便,因为大家普遍认为官方发布的量化版本更具权威性。
For officially released LLMs, it is suggested that awq and gptq quantized versions b…
-
Since security is a priority and data sharing in some cases in not an option. I was thinking about using local hosted open source LLMs.
Is that achievable and which LLMs do you recommend?
wtfc2 updated
2 weeks ago
-
Hi, I discovered your plugin yesterday and found it quite interesting.
I found out that I can start a local server with AI models with lmstudio.ai, after setting that up, I wanted to use it from my…
-
Is there Support for the Openai Whisper models or only LLMs so far?
-
Bug Description
Even though I am following the latest documentation . I am still not able to import llama index.
**Issue 1**
Nvidia jetson container link:https://github.com/dusty-nv/jetson-contai…
-
In [continuous_eval/metrics/generation/text/llm_based.py](https://github.com/relari-ai/continuous-eval/blob/main/continuous_eval/metrics/generation/text/llm_based.py), line 39, the full model paramete…
-
### Bug Description
Llama is not working with latest langchain using llama-index-llms-langchain package and because of this there is an issue with using tiktoken 0.7.0 that is required for gpt-4o
…
-
Allow to specify URL to use to run it with local LLMs like Ooga Booga.
-
If the use of OpenAI is not possible due to privacy concerns, it would be beneficial to have the option to utilize a locally installed language model.
One possible solution is to allow for customiz…