train "monorec_depth_ref" problem

[zoey-phys]

Question 1:

• When I run the last training python train_monorec.py --config configs/train/monorec/monorec_depth_ref.json --options stereo stereo_repr it can't be continued. Train Epoch: 1 [12800/13714 (93%)] Loss: 0.340444 Loss_dict: {'sdl_0': tensor(0.0036, device='cuda:0'), 'static_md2l_0': tensor(0.0692, device='cuda:0'), 'dynamic_md2l_0': tensor(0.1163, device='cuda:0'), 'md2l_0': tensor(0.0693, device='cuda:0'), 'sdl_1': tensor(0.0036, device='cuda:0'), 'static_md2l_1': tensor(0.0695, device='cuda:0'), 'dynamic_md2l_1': tensor(0.1168, device='cuda:0'), 'md2l_1': tensor(0.0696, device='cuda:0'), 'sdl_2': tensor(0.0036, device='cuda:0'), 'static_md2l_2': tensor(0.0702, device='cuda:0'), 'dynamic_md2l_2': tensor(0.1136, device='cuda:0'), 'md2l_2': tensor(0.0703, device='cuda:0'), 'sdl_3': tensor(0.0039, device='cuda:0'), 'static_md2l_3': tensor(0.0722, device='cuda:0'), 'dynamic_md2l_3': tensor(0.1166, device='cuda:0'), 'md2l_3': tensor(0.0722, device='cuda:0'), 'loss': tensor(0.3404, device='cuda:0')} hold on here

Question 2:

• Could you provide your GPU configuration and training time?

[Brummi]

Hi, sorry for taking so long to reply! (Have to submit my MSc thesis in a few days) I ran into a similar issue at some point, but for me the problem was that my data loaders got stuck and were not able to return data anymore. I would assume that you have a similar issue. Could you maybe try out a different number of workers for the data loader and let me know how it goes?

[Brummi]

Regarding your previous quesitons: The poses data should be in {}/poses_dso/*.txt

We used 12GB Titan Xs for training, and also tested it on 24GBs GPUs

[zoey-phys]

Congratulations on your graduation. Thank you so much, I understand those two problems. If I want to train the indoor scenes, is there a suitable dataset?

[nynyg]

The ScanNet (http://www.scan-net.org/) dataset is a quite popular dataset for real indoor scenes. For the synthetic dataset, you may want to check the Replica dataset (https://github.com/facebookresearch/Replica-Dataset).