-
I load the model to GPU like this:
```python
llm = AutoModelForCausalLM.from_pretrained("LLM-model",
model_file="vinallama-7b-chat_q5_0.gguf",
…
-
I just used base script from the TheBloke repo and got an error during loading the model:
OSError: libcudart.so.12: cannot open shared object file: No such file or directory
my script is :
from…
-
-
I try to pass the arguments that are listed in the documentation, but I get nowhere.
handler = StdOutCallbackHandler()
llm = CTransformers(model='TheBloke/MPT-7B-Instruct-GGML',model_file='mpt-7b…
-
- [ ] [unsloth/README.md at main · unslothai/unsloth](https://github.com/unslothai/unsloth/blob/main/README.md?plain=1)
# unsloth/README.md at main · unslothai/unsloth
…
-
Hello guys, i try to run mpt-7b model , and i am getting this code, i appreciate any help, here is the detail
Node.js v19.5.0
node_modules\llama-node\dist\llm\llama-cpp.cjs:82
this.inst…
-
I am trying to run llama2 gguf on windows 11 Version 22H2. I have python 3.11 installed on my local machine.
Below is the code:
```
import gradio as gr
from ctransformers import AutoModelForCausal…
-
Hi,
is there already a feature to start this in something like a Server-Mode? I want to use starcoder as an endpoint for the VSCode-Addon [HF Code Autocomplete](https://marketplace.visualstudio.com…
-
Hello, from what I see and understand from the CMakeLists.txt, the current version is compatible with cuda 12.0 and greater. Would it be safe if I compile using 11.8?
Thanks!
-
- [ ] [README.md · defog/sqlcoder-7b-2 at main](https://huggingface.co/defog/sqlcoder-7b-2/blob/main/README.md?code=true)
# README.md · defog/sqlcoder-7b-2 at main
**DESCRIPTION:**
```yaml
license:…