search

wangzy22 / TAP

[ICCV 2023] Take-A-Photo: 3D-to-2D Generative Pre-training of Point Cloud Models
MIT License
40 stars 1 forks source link

Multi-GPUs Runtime Error #3

Open RyanG41 opened 1 year ago

RyanG41 commented 1 year ago

Hi, Thanks for the woderful job. I encountered a error caused by distributed training, maybe? I ran the code on multi-gpus and got the error below: RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by passing the keyword argumentfind_unused_parameters=Truetotorch.nn.parallel.DistributedDataParallel,..... In the train.py I see the code for multi processing, but here I dont know how to fix it, or can I force the code to run on only 1 gpu? Thanks for the help of any kind you provide.

eriche2016 commented 11 months ago

@RyanG41 same issue here, have u solved this issue?