search

eth-ait / MultiPly

MultiPly: Reconstruction of Multiple People from Monocular Video in the Wild (CVPR2024 Oral)
https://eth-ait.github.io/MultiPly/
194 stars 18 forks source link

Enable multi GPU's causes errors #8

Closed tweezlednutball closed 1 month ago

tweezlednutball commented 2 months ago

devices=-1, in train.py with a machine with 2 more more GPU's causes these errors: RuntimeError: It looks like your LightningModule has parameters that were not used in producing the loss returned by training_step. If this is intentional, you must enable the detection of unused parameters in DDP, either by setting the string value strategy='ddp_find_unused_parameters_true' or by setting the flag in the strategy with strategy=DDPStrategy(find_unused_parameters=True).

When I add the string value `strategy='ddp_find_unused_parameters_true': RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! (when checking argument for argument mat2 in method wrapper_CUDA_bmm)

jzr99 commented 2 months ago

Hi,

Currently, our method does not support multiple GPU training. Set export CUDA_VISIBLE_DEVICES=0 to choose gpu id before training.