-
**LocalAI version:**
`quay.io/go-skynet/local-ai:v1.22.0-cublas-cuda11`
**Environment, CPU architecture, OS, and Version:**
`Linux glados 6.2.0-26-generic #26-Ubuntu SMP PREEMPT_DYNAMIC Mon…
-
Hi, I see the mention of running this model on llama.cpp in README. Did you get a manage to get it to run and quantize with good output? I'm trying to evaluate if this model can be used for speculativ…
-
### System Info
- `transformers` version: 4.28.0.dev0 (656e869a4523f6a0ce90b3aacbb05cc8fb5794bb)
- Platform: Linux-5.15.0-67-generic-x86_64-with-glibc2.35
- Python version: 3.10.10
- Huggingface_h…
-
### Description
```shell
branch: main
```
### Reproduced Steps
```shell
To reproduce this issue:
pip3 install torch==1.9.1+cu111 -f https://download.pytorch.org/whl/torch_…
-
The same issue already discussed in #98, but it is not resolved despite that issue being closed with no obvious resolution.
When following the guide here https://docs.coreweave.com/compass/examples…
-
StableLM looks GPTNeoX and has `query_key_value` parameters.
I thought I could apply LoRA to StableLM by specifying `target_modules='query_key_value'`, but I got the following error.
```
Tracebac…
-
## Feature request
When using a model that uses `gradient_checkpointing` and if a user wants to call `generate` with `use_cache`, it leads some models to bugs, such as the one described in https://…
-
**LocalAI version:**
v1.17.0
**Environment, CPU architecture, OS, and Version:**
Both docker and standalone, M1 Pro Macbook Pro, MacOS Ventura 13.4
**Describe the bug**
It seems it is…
-
**LocalAI version:**
latest
**Environment, CPU architecture, OS, and Version:**
amd64 thinkpad + kind
**Describe the bug**
We can see localai receives the prompts buts fails to respond to…
-
### System Info
Latest transformers main branch, Python 3.10
### Who can help?
@sgu
### Information
- [ ] The official example scripts
- [ ] My own modified scripts
### Tasks
- […