-
### System Info
Version: `ghcr.io/huggingface/text-generation-inference:latest`
Model: `TheBloke/MythoMax-L2-13B-GPTQ:gptq-4bit-128g-actorder_True`
GPU: Nvidia A4000
Ubuntu 22.04
### Issue
…
-
### System Info
- `transformers` version: 4.29.0
- Platform: macOS-12.2.1-arm64-arm-64bit
- Python version: 3.9.16
- Huggingface_hub version: 0.14.1
- Safetensors version: not installed
- PyTo…
-
### System Info
2023-05-15 21:13:50.400043: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
WARNING:tensorflow:From /usr/local/lib/python3.10/dist-pac…
-
There is a bug in gpt_tokenize function in examples/common.cpp
Followings are comparison with a huggingface tokenizer,
```
# huggingface
>>> from transformers import AutoTokenizer, AutoModelFo…
-
Does not work despite inserting the key from huggingface
-
**Describe the bug**
While trying to quantise model https://huggingface.co/GeorgiaTechResearchInstitute/starcoder-gpteacher-code-instruct (`GPTBigCodeForCausalLM`) I get many instances of this error:…
-
**LocalAI version:**
`quay.io/go-skynet/local-ai:master-cublas-cuda12-core`
**Environment, CPU architecture, OS, and Version:**
`Linux user-Z68X-UD3P-B3 6.2.0-39-generic #40~22.04.1-Ubunt…
-
**LocalAI version:**
V1.21
root@63429046747f:/build# ./local-ai --version LocalAI version 4548473 (4548473acf4f57ff149492272cc1fdba3521f83a) llmai-api-1 | 3:04AM DBG Loading model '
**Environment, C…
-
Hi and thanks for your awesome backend.
I was just wondering if you can add those options to your generate method:
repeat_penalty
repeat_last_n
They are standard GPT parameters and your backend …
-
**LocalAI version:**
`quay.io/go-skynet/local-ai:v1.22.0-cublas-cuda11`
**Environment, CPU architecture, OS, and Version:**
`Linux glados 6.2.0-26-generic #26-Ubuntu SMP PREEMPT_DYNAMIC Mon…