对sft阶段的数据进行packing之后，同一条训练数据内的指令之间是否会相互影响？ - Githubissues

hiyouga / LLaMA-Factory

Unify Efficient Fine-Tuning of 100+ LLMs

Apache License 2.0

25.26k stars 3.13k forks source link

对sft阶段的数据进行packing之后，同一条训练数据内的指令之间是否会相互影响？ #4462

Closed Zheng-Jay closed 3 days ago

Zheng-Jay commented 3 days ago

Reminder

[X] I have read the README and searched the existing issues.

System Info

如题，同一条训练数据内的指令之间能否看到彼此，会不会相互影响呢？

Reproduction

无

Expected behavior

No response

Others

No response