-
Hi @molly-smith, we are recently enabling LLM on CPU and want to use AutoTP to reduce latency of LLM inference. From our experiment, AutoTP can work with GPT-J 6B inference well, but does not suppor…
-
I've just swapped over the endpoints in my code, and the parsing logic broke for streaming responses due to the lack of `data: ` before the JSON object. Is this intended behavior, for some reason?
…
-
### System Info
The error as specified in the title occurs with various nvidia/cuda configurations 530/12.1 530/11.8 470/11.4 etc. and with various version of pythia (sft-1 and sft-4) and starcoder. …
-
i fine tune bloomz-7b1 using lora config
after fine tuning and get adapter config
i got error when try generate text from it
heres sample code
```
model_name_or_path = "bigscience/bloomz-…
-
**LocalAI version:**
728f297bb8c390d0de7b95b1969cbf1ad1864ab4
**Environment, CPU architecture, OS, and Version:**
arm64(mac m2 pro) Ventura 13.3.1
**Describe the bug**
make build f…
-
We have encountered a significant issue with the ZeroShotClassifier, as it is consistently returning inaccurate results when provided with example prompts from the [docs.](https://skorch.readthedocs.i…
-
**LocalAI version:**
v1.13.0 commit ffaf3b1d362cc4d0045e1ed60bbcf0481d21f8ef
**Describe the bug**
I changed `make build` to `make GO_TAGS=stablediffusion build` in `Dockerfile` and during the bui…
-
大神你好，感谢开源工作，我现在尝试使用BLOOMZ-1.7B作为基础模型（bin文件3.44GB）来finetune一个对话模型，使用的是train_scripts/single_gpu/run_FT.sh，我的GPT是32GB配置的V100，请问还是不够训练吗？我看一直报错cuda out of memory。
I am trying run the supervised fine tuning in Google Colab. Al…
-
Sorry for stupid question, but I am totally newbie in Docker and using Huggingface locally (not via colab or something else). This is the command to model for the first time , for example:
docker r…