-
batch inference
-
Similar to [this](https://github.com/abetlen/llama-cpp-python) repository for [llama-cpp](https://github.com/ggerganov/llama.cpp), is anyone aware of efforts to do the same for starcoder-cpp?
More …
-
Hey,
I wanted to test the Mistral-7b model and tried to run the following code:
from ctransformers import AutoModelForCausalLM, AutoConfig, Config
conf = AutoConfig(Config(temperature=0.8, repe…
-
Whenever I try to load a QLoRA-merged Falcon 40B model, the error below happens.
`error loading model: falcon.cpp: tensor 'lm_head.weight' is missing from model`
The hacky way I did to resolv…
-
- [ ] [README.md · defog/sqlcoder-7b-2 at main](https://huggingface.co/defog/sqlcoder-7b-2/blob/main/README.md?code=true)
# README.md · defog/sqlcoder-7b-2 at main
**DESCRIPTION:**
```yaml
license:…
-
@talhaanwarch nice job on the app! Looks great.
- Can this run on GPU as well as CPU? This code appears to run WAY faster than `llama-cpp-python` bindings.
- Curious to ask if you would be inte…
-
I have been trying to get llama 2 models to function correctly. They start off ok but then all of them goes into a loop with repetitions or gibberish.
I haven't tried setting model_type:llama to s…
-
Hi I am trying to call "from gpt4allj.langchain import GPT4AllJ", but i get that error
'''ModuleNotFoundError: No module named 'langchain.callbacks.manager'''
I have istalled langchain + gpt4lallj …
-
## Expected Behavior
The prompt for typing and interacting with the AI should be there
## Current Behavior
The prompt is gone, including the sidebar for changing models and personalities.
## S…
d13g4 updated
9 months ago
-
@mayooear thanks for this nice project. Very helpful. Have a couple of items:
1. The current setup is missing a required package: sentence_transformers
2. Any thoughts on how to run this in GPU?