-
Hi YuYang,
Congratulations on your great work! It would be really nice if you can upload the model to Hugging Face hub.
This would help model discovery and integration with tools.
For examp…
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
```py
(base) root@ubuntu:/data/ChatGLM2-6B# python cli_demo.py
Compile p…
-
-
chatglm2模型的原始大小12G左右,在T4卡(显存15G)上可以单卡加载起来。
使用fastllm.utils.convert转成fastllm之后,llm文件有24G,fastllm.create_llm(model_path)加载时候直接失败了
放在多卡上计算速度就比原始chatglm2在单卡上慢多了
-
when i try to test tensor_parrallel with chatglm2-6B, something went wrong.
![image](https://github.com/NVIDIA/TensorRT-LLM/assets/83857604/737df276-cbd6-49d0-938f-2d2ba594cc6c)
-
As the title says, on the same graphics card (3090), CodeGeeX2-6b is much slower than ChatGLM-6b. According to the official demo, I would like to know if there are any tricks.
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
采用数据并行lora微调,报错如下
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so.11…
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
部署环境:阿里云ECS
CPU&内存:12核(vCPU) 92 GiB
操作系统:Anolis OS 8.6 RHCK 64位
GPU: NVIDIA V1…
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
/home/chenjk/miniconda3/envs/chatglm2/lib/python3.9/site-packages/transformers/optimization.p…
-
https://github.com/THUDM/ChatGLM2-6B/issues/535