-
i use this model:
bartowski/Mistral-Nemo-Instruct-2407-GGUF
lmstudio Version 0.3.3 (Build 1)
i apply the jinja template:
{%- if messages[0]["role"] == "system" %}
{%- set system_m…
-
When trying to utilize the full context size for this model https://huggingface.co/vsevolodl/Llama-3-70B-Instruct-Gradient-1048k-GGUF i get an out of RAM(?) error like this:
```
{
"title": "Fai…
-
**Is your feature request related to a problem? Please describe.**
Yes and no, it would be nice to be able to use the extension on the web version of vscode.
**Describe the solution you'd like**
…
-
I have been testing different models that are “ready to process PDFs” from HuggingFace, I have installed them in different tools and I have tested different models from some 7B parameters up to 32B pa…
-
### How are you running AnythingLLM?
AnythingLLM desktop app
### What happened?
When requesting/api/v1/openai/chat/completions for anything-llm 1.6.0 desktop version, the max_token does not take ef…
-
I'm seeing a weird behavior with vision models.
I am using the Default LM Studio Windows config, which is the only one I have been able to get vision models to work with.
I have tried 2 differ…
-
I have tried a variety of models that support function calls, none of them gives the correct answer using LM Studio. Since it uses an 'API compatible with that of OpenAI it should accept the `tools` k…
-
Related to #34.This would be the holy grail for this maintainer, but implied in this is that we would need a _trained_ local model, since local models are terrible with the tasks we need unless fine-t…
-
Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
CUDA_VISIBLE_DEVICES=0 ./llama-server --host 0.0.0.0 --port 8008 -m /home/kemove/model/gemma-2-27b-it-Q5_K_S.gguf -ngl 99 -t 4 -n…
-
Uncaught Exception:
Error: dlopen(/Applications/LM Studio.app/Contents/Resources/app/.webpack/main/build/Release/gguf/llm_engine.node, 0x0001): Symbol not found: _cblas_sgemm$NEWLAPACK$ILP64
Referen…