-
### Is there an existing issue for this bug?
- [X] I have searched the existing issues
### 🐛 Describe the bug
I failed to run ChatGLM model with ColossalAI 0.3.6.
backtrace is here
----…
-
https://github.com/bytedance/ByteMLPerf/blob/main/byte_infer_perf/llm_perf/workloads/chatglm2-torch-fp16-6b.json
We run on A100-40G to get output logits with the below configuration:
```json
{
…
-
Requirement already satisfied: torch in c:\programdata\anaconda3\lib\site-packages (from flash-attn==2.0.4) (2.1.0.dev20230721+cu121)
Requirement already satisfied: einops in c:\programdata\anaconda3…
-
报错信息
python3 -m ftllm.chat -t 16 -p ~/llm/fastllm/models/ --dtype int4
Load AutoTokenizer failed. (you can try install transformers)
Try load fastllm tokenizer.
zsh: segmentation fault…
-
老哥,执行了sh tran.sh pre_tran.py,修改成了单卡模式,为啥会出现下面这个错误,搜了一下好像是“模型或数据未正确移至相应设备:”。
Number of trainable parameters = 1,431,996,416
0%| …
-
### 请提出你的问题
- 前提条件:单机单卡已经跑通chatglm2的lora微调训练代码;llama的多卡pp并行训练已跑通。
- 问题场景:想进一步尝试单机多卡,设置/chatglm2/lora_argument.json配置文件中的 "pipeline_parallel_degree": 4,然后参照官网样例,启动命令行:`srun --gres=gpu:4 python3 -u …
-
[2023-08-12 01:22:11,409] [INFO] [logging.py:96:log_dist] [Rank 0] DeepSpeed info: version=0.10.0, git-hash=unknown, git-branch=unknown
Traceback (most recent call last):
File "/root/inpc_projects…
-
### 软件环境
```Markdown
- paddlepaddle-gpu: 0.0.0.post118
- paddlenlp: 2.8.0.post0
```
### 重复问题
- [X] I have searched the existing issues
### 错误描述
```Markdown
硬件:ai studio V100 32G
…
-
Hello! My project compiled perfectly with pyinstaller, but the exe file was slow to start, so I started trying to compile with Nuitka, but I ran into problems.
I used nuitka to compile the project e…
-
Chatglm2-6b contains multiple LLM model files
![image](https://github.com/go-skynet/LocalAI/assets/31379313/7b00a329-8084-45ed-a443-a476b4c9b57e)