extract environment initialization

mindspore-lab / mindone

one for all, Optimal generator with No Exception

https://mindspore-lab.github.io/mindone/

Apache License 2.0

338 stars 63 forks source link

Closed hadipash closed 4 months ago

hadipash commented 5 months ago

This PR introduces the extraction of the training environment initialization, as it is utilized across the project. I have also added a graph caching feature, which is exceptionally helpful during the hyperparameters tuning stage. Here are the numbers (tested on SDv2.1 and pokemon-blip dataset):	Device	Environment	First epoch time without caching	With caching
910	MS2.1	15+ minutes	4~ minutes
910*	MS2.2.1	17+ minutes	5+ minutes

zhtmike commented 5 months ago

@HaoyangLee the graph cache is fixed now?

HaoyangLee commented 5 months ago

@HaoyangLee the graph cache is fixed now?

Haven't used that feature for a while. Not sure about it in the latest MS version. @hadipash 's testing result looks nice.