issues
search
baichuan-inc
/
Baichuan-7B
A large-scale 7B pretraining language model developed by BaiChuan-Inc.
https://huggingface.co/baichuan-inc/baichuan-7B
Apache License 2.0
5.67k
stars
506
forks
source link
[Question] 请问继续预训练的loss降到什么水平是合格的
#115
Open
parkLGW
opened
1 year ago
parkLGW
commented
1 year ago
Required prerequisites
[X] I have read the documentation
https://github.com/baichuan-inc/baichuan-7B/blob/HEAD/README.md
.
[X] I have searched the
Issue Tracker
and
Discussions
that this hasn't already been reported. (+1 or comment there if it has.)
[X] Consider asking first in a
Discussion
.
Questions
用自有数据继续预训练,loss一直在2.1左右,请问是正常的吗
Checklist
[X] I have provided all relevant and necessary information above.
[X] I have chosen a suitable title for this issue.
Required prerequisites
Questions
用自有数据继续预训练,loss一直在2.1左右,请问是正常的吗![image](https://github.com/baichuan-inc/Baichuan-7B/assets/35214930/037a9e12-45ac-4d81-8109-d911233d28f0)
Checklist