threestudio-project / threestudio

A unified framework for 3D content generation.
Apache License 2.0
6.32k stars 480 forks source link

fix multigpu #212

Closed guochengqian closed 1 year ago

guochengqian commented 1 year ago
seed_everything(cfg.seed, workers=True) # using same seed to init model weights

dm = threestudio.find(cfg.data_type)(cfg.data)
system: BaseSystem = threestudio.find(cfg.system_type)(
    cfg.system, resumed=cfg.resume is not None
)
system.set_save_dir(os.path.join(cfg.trial_dir, "save"))

seed_everything(cfg.seed + get_rank(), workers=True) # using different seed now for loading different data
bennyguo commented 1 year ago

Thanks @guochengqian ! I'll verify this ASAP.

bennyguo commented 1 year ago

Hi @guochengqian ! It turns out we don't need to call seed_everything twice -- Lightning will handle the synchronization of model params across ranks. I opened a new PR with minor fixes and will close this now.