[Question] 关于数据处理的疑问

[mynewstart]

Required prerequisites

• [X] I have read the documentation https://github.com/baichuan-inc/baichuan-7B/blob/HEAD/README.md.
• [X] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• [X] Consider asking first in a Discussion.

Questions

HI, 现在代码对于数据处理的方式是直接拼接text到max_length，中间用eos。这样操作的话在计算attention的时候，text2其实可以看到text1的内容，如果两个text之间没有啥联系的话会有影响吗？你们在实践中是会mask掉text1的token还是说每个text的文本尽可能的长呢，一个样本只有一个text?

Checklist

• [X] I have provided all relevant and necessary information above.
• [X] I have chosen a suitable title for this issue.