Closed zhouyu5 closed 5 days ago
This is a follow-up PR for issue #9455 .
Say we want to lauch training on two XPUs:
Before fixing Data will be downloaded twice and process twice, and causing conflict, see below:
After fixing Data will be downloaded and processed only once on main process, see below:
This is a follow-up PR for issue #9455 .
Say we want to lauch training on two XPUs:
Before fixing Data will be downloaded twice and process twice, and causing conflict, see below:
After fixing Data will be downloaded and processed only once on main process, see below: