-
### Describe the Question
Please provide a clear and concise description of what the question is.
Running tokenizer on dataset (num_proc=10): 0%| …
-
# 结论
灾难性遗忘,增量预训练起了反效果
### 领域
文言文
### 预训练和微调数据集
预训练:[daizhige](https://github.com/garychowcmu/daizhigev20)
微调:https://github.com/NiuTrans/Classical-Modern
### 实验结果
具体数据我就保密了,只说一下相对结果
指标:…
-
!python supervised_finetuning.py \
--model_type llama \
--model_name_or_path ./merged-pt \
--train_file_dir ./data/finetune \
--validation_file_dir ./data/finetune \
--per_dev…
-
### Describe the Question
Please provide a clear and concise description of what the question is.
-
Loading LoRA for causal language model
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
Loading checkpoint shards: 0%| …
-
This [hardware requirement section](https://github.com/shibing624/MedicalGPT/blob/main/README_EN.md#hardware-requirement) isn't clear
1. Does this table show RAM or hard disk space? Or is it VR…
-
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/share/huada/home/biwenshuai/zxl_2024/RELLM/MedicalGPT/supervised_finetuning.py…
-
下面是执行过程和问题说明,请帮忙看看,谢谢。
一、问题说明:
1. 使用: “run_training_dpo_pipeline.ipynb”的副本,在google colab运行
2. 替换模型:
--model_type auto \
--model_name_or_path Qwen/Qwen1.5-0.5B-Chat \
3. 过程:PT,SFT训练正常,merge正…
-
### Describe the Question
https://github.com/shibing624/MedicalGPT/blob/main/supervised_finetuning.py#L871
请问为什么全参sft时需要将模型转为fp32?
-
可能和执行变量 --model_name_or_path merged-pt 有关系 ,这样的写法在colab 执行没问题,但是本地服务器就出现下面的错。
这样的写法 --model_name_or_path ./merged-pt 也报同样的错误。
----------
INFO | __main__:main:1448 - Saving model checkpo…