-
跑https://colab.research.google.com/github/shibing624/MedicalGPT/blob/main/run_training_dpo_pipeline.ipynb#scrollTo=J5kYehpzESyt (run_training_dpo_pipeline.ipynb)这个脚本时,pretrain阶段报错
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
https://github.com/shibing624/MedicalGPT
参考这个项目,预训练,指令微调,rm模型训练,ppo都有现成的。
### Expected Beha…
-
### 先决条件
- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-com…
-
我试图用
CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node 2
这样并行训练,会直接报错
```
ValueError: You can't train a model that has been loaded in 8-bit precision on multiple devices in any distributed mode.…
-
"本项目开源了基于ChatGLM-6B LoRA 16-bit指令微调的中文医疗通用模型。模型呢?
按照快速提示提供的步骤无法运行.
laszo@LAPTOP-6MNNHCID:~$ . myvenv/bin/activate
(myvenv) laszo@LAPTOP-6MNNHCID:~$ cd /mnt/d/dev/code/MedicalGPT-zh/
(myvenv) l…
-
推断没问题,是否支持训练微调?(我尝试时,尚不支持)
-
### Is your feature request related to a problem? Please describe.
_No response_
### Solutions
如何在领域数据上二次预训练
### Additional context
_No response_
-
你好徐老师,我使用的是windows四卡3090机器,每张显卡24G显存,因为是win平台,就写了一个bat脚本来运行
`@echo off
set CUDA_VISIBLE_DEVICES=0,1,2,3
call python supervised_finetuning.py ^
--model_type baichuan ^
--model_name_or_p…
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
有看到llama支持continual pretraining,想问一下chatglm能支持这种继续预训练方式吗?
### Expected Behavior
_No respons…
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
目前只有指令微调,指令微调需要有( 指令,回答),但是它并不是语言模型本身的训练。
### Expected Behavior
https://github.com/THUDM/Ch…