-
请教下数据处理部分,tokenizer分词后的position_ids是怎么生成的
def tokenize_wo_pad_function(examples):
tokenized_examples = tokenizer(examples["patent"])
print("tokenized_examples00",type(tokenized_…
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
ptuning对于想让模型掌握特定领域知识的表现不太好,怎么根据数据集进行全量调参或者二次预训练呢?
### Expected Behavior
_No response_
###…
-
### Is your feature request related to a problem? Please describe.
_No response_
### Solutions
求预训练数据格式
### Additional context
_No response_
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
我想在ChatGLM-6B上做进一步的pretrain。
请教一下ChatGLM-6B使用的网络架构是否和GLM完全一样。
GLM论文显示,pretrain的输入应该是p…
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
你好,我有一些对话数据,这些数据是打了分数的,我想问一下如何用我手上的数据进行“人类反馈强化学习”微调?我会尝试 P-Tuning v2,但更想尝试“人类反馈强化学习”的微调…
-
Thanks for the great repo
i have two questions about training the models (specifically WizardCoder):
1. have you tried training with QLoRa, and not just LoRa ? are you considering adding it to t…
mrT23 updated
9 months ago
-
你好,当训练环境是AMD ROCM环境时,执行run_pt.sh会报错,错误如下:
RuntimeError: HIP error: invalid argument
HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorr…
-
您好,没成功发送您的邮箱,冒昧的在这问一下:我在https://github.com/shibing624/MedicalGPT/issues/46看到了您的回复,看到了您使用deepspeed stage3成功跑通了代码。我直接将deepspeed_config.json中的stage修改为3运行, 报下面这个错误:
DeepSpeed Zero-3 is not compatible wit…
-
### Is your feature request related to a problem? Please describe.
数据集的格式都是输入问答对的方式,能不能直接输入一篇文档作为数据集来微调训练?
比如我有一个法条的txt文档,一万字左右。我想塞进去直接训练,让模型理解。然后对模型提问相关的问题,让他回答法条问题。
类似于chatpdf这种,但是跟chatpdf不一样,cha…
-
### Is your feature request related to a problem? Please describe.
_No response_
### Solutions
https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki/%E9%A2%84%E8%AE%AD%E7%BB%83%E8%84%9A%E6%9C%AC
### …