为什么用4K模型训练时loss是0？

InternLM / InternLM-XComposer

InternLM-XComposer2 is a groundbreaking vision-language large model (VLLM) excelling in free-form text-image composition and comprehension.

1.92k stars 121 forks source link

Closed zweiqi closed 2 months ago

yuhangzang commented 2 months ago

This is due to the incorrect SFT data format. You can set a breakpoint to view the value of the text variable in here to verify.

yuhangzang commented 2 months ago

Kindly re-open if you still have any issues.