-
i saw the issue with chatglm2-6b.
it run successfully if with numactl -m 0 -C 0-23.
it run failed if with numactl -m 0 -C 0-31, or 0-47 , or 0-55.
i can be reproduced with INT8_ASYM or 4BIT_…
-
我将chatglm2下载到如下图的目录
![image](https://github.com/user-attachments/assets/cabf6b33-475a-4783-b74e-7bb26fdc1088)
但报错说没有指定的模型
-
### 错误描述
机器: h100-80g-hbm3
基于以下的chatglm-6b-xxx.json配置测试,在tp1, bs24, inputlen1024下报OOM
![image](https://github.com/user-attachments/assets/cd7eb052-342f-4eb1-ae92-70b55f0a89a6)
![image](https://g…
-
1. https://github.com/THUDM/ChatGLM-6B/blob/main/README_en.md
> ChatGLM-6B uses technology similar to ChatGPT, optimized for Chinese QA and dialogue. The model is trained for about 1T tokens of Chine…
-
Requirement already satisfied: torch in c:\programdata\anaconda3\lib\site-packages (from flash-attn==2.0.4) (2.1.0.dev20230721+cu121)
Requirement already satisfied: einops in c:\programdata\anaconda3…
-
https://github.com/bytedance/ByteMLPerf/blob/main/byte_infer_perf/llm_perf/workloads/chatglm2-torch-fp16-6b.json
We run on A100-40G to get output logits with the below configuration:
```json
{
…
-
Chatglm2-6b contains multiple LLM model files
![image](https://github.com/go-skynet/LocalAI/assets/31379313/7b00a329-8084-45ed-a443-a476b4c9b57e)
-
when I convert chatglm2-6b with A10, there is error as below:
Traceback (most recent call last):
File "/code/tensorrt-llm/tensorrt-llm/TensorRT-LLM/examples/chatglm/build.py", line 895, in
r…
-
[](https://github.com/THUDM/ChatGLM2-6B)
ghost updated
7 months ago
-
### Checked other resources
- [X] I searched the Codefuse documentation with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.
- [X] I am sure that …