-
### System Info
Transformers=4.31
Torch=2.01
Cuda=11.8
Python=3.10
A100 GPU 80GB
### Who can help?
@ArthurZucker , @younesbelkada , @gante
### Information
- [ ] The official example script…
-
**Describe the bug**
Following #2547 I tried to run the model gpt-neoxt-chat-base-20b, which is a neox-20B derivative I think and I think it should work.
Inference works if the model is loaded the n…
-
现在很多中文用户正在基于chatglm系列模型搭建自己的langchain应用,而且chatglm.cpp项目已提供ggml支持,按理说这个应该也属于native_int4的范畴,所以能否支持一下bigdl.llm.langchain在这系列模型上的BigdlNativeEmbeddings API呀?
-
import os
import pickle
from typing import List
from dataclasses import field, dataclass
from utils import set_default_to_empty_string
FOLDER_ROOT = (
os.path.abspath(os.path.dirname(os.pa…
-
**LocalAI version:**
According to git the last commit is from Sun Sep 3 02:38:52 2023 -0700 and says "added Linux Mint"
**Environment, CPU architecture, OS, and Version:**
Linux instance-7 6.…
-
I have 4 GPUs and 3 models called small, medium and large. I want to deploy small model on GPU 0, medium model on GPU 1, and large model on GPU 2 and GPU3 with tensor_para_size=2 due to large model is…
-
## Description
Finetune T5 and GPTNeoX using HateCheck data.
## Steps
- [x] Experiment fine-tuning already pre-trained T5 tasks such as "summarize:" but using hateful data.
- [x] Try other p…
-
When inspecting the config of the hybrid model https://huggingface.co/state-spaces/mamba2attn-2.7b/blob/main/config.json, I came up with two questions:
- Why is the number of heads 30? Wouldn't we us…
vasqu updated
2 months ago
-
**LocalAI version:**
#895
**Environment, CPU architecture, OS, and Version:**
sh-5.2$ uname -a
MSYS_NT-10.0-19045 DESKTOP-S7HQITA 3.4.7-ea781829.x86_64 2023-07-05 12:05 UTC x86_64 Msys
…
-
I followed the settings to set the value display limit to `1024`, but the actual values displayed do not match the size of the weights (`arg0_1`):
This is from the quick_start example colab.