Error local_test_LBA.sh

[ylee1123]

I tried "sh local_test_LAB.sh" but couldn't get the result. first, when I tried command below: python run_dso.py --dso_path ../third_party/dso/build/bin/dso_dataset --data_fldr ../data/7scenes_office_seq_01 --cam_info_file DSO/cam_info_7scenes.mat --res_path ./dso_res --name_pattern .color --maxframe 300

the output was something like this. nan -> 4.869459 4.819004 7.189675 nan nan RE-TRACK ATTEMPT 82 with initOption 82 and start-lvl 2 (ab -0.082485 5.409135): 4.862892 4.789245 7.140387 nan nan -> nan nan 19.306097 nan nan ii: 144, idsToPlay: 300 ii: 145, idsToPlay: 300 ii: 146, idsToPlay: 300 ii: 147, idsToPlay: 300 ii: 148, idsToPlay: 300 RE-TRACK ATTEMPT 1 with initOption 1 and start-lvl 2 (ab -0.101255 8.301172): 8.238788 6.501144 7.314152 nan nan -> 8.239683 6.502624 7.327506 nan nan

then when I tried command below: CUDA_VISIBLE_DEVICES=0 python3 test_KVNet_LBA.py --exp_name LBA_demo --sigma_soft_max 10 --t_win 2 --d_min .1 --d_max 5. --feature_dim 64 --ndepth 64 --dataset single_folder --dataset_path ../data/7scenes_office_seq_01 --img_name_pattern *.color.png --intrin_path ./DSO/cam_info_7scenes.mat --LBA_max_iter 20 --LBA_step 0.01 --opt_r 0 --opt_t 1 --use_dso_R --opt_next_frame --min_frame_idx 0 --max_frame_idx 298 --refresh_frames 300 --model_path ./saved_models/kvnet_scannet.tar

it gave me this error: Parsing the arguments... Initnializing the KV-Net bn_running_avg = 0 RefineNet_UNet2D: init conv2d RefineNet_UNet2D: init conv2d init transposed 2d RefineNet_UNet2D: init conv2d RefineNet_UNet2D: init conv2d init transposed 2d RefineNet_UNet2D: init conv2d RefineNet_UNet2D: init conv2d RefineNet_UNet2D: init conv2d KV-Net initialization: with R-net: True refinement name: DPV loading KV_net at ./saved_models/kvnet_scannet.tar Done Getting the paths for traj_0 Done init initial pose from DSO estimations ... DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return frame_cnt :0, include invalid poses updating array ... done DSO did not return frame_cnt :1, include invalid poses updating array ... done DSO did not return frame_cnt :2, include invalid poses updating array ... done DSO did not return frame_cnt :3, include invalid poses updating array ... done DSO did not return frame_cnt :4, include invalid poses updating array ... done DSO did not return frame_cnt :5, include invalid poses updating array ... done DSO did not return frame_cnt :6, include invalid poses updating array ... done DSO did not return frame_cnt :7, include invalid poses updating array ... done DSO did not return frame_cnt :8, include invalid poses updating array ... done DSO did not return frame_cnt :9, include invalid poses updating array ... done DSO did not return frame_cnt :10, include invalid poses updating array ... done DSO did not return frame_cnt :11, include invalid poses updating array ... done DSO did not return frame_cnt :12, include invalid poses updating array ... done DSO did not return frame_cnt :13, include invalid poses updating array ... done DSO did not return frame_cnt :14, include invalid poses updating array ... done DSO did not return frame_cnt :15, include invalid poses updating array ... done DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return DSO did not return THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=405 error=11 : invalid argument Traceback (most recent call last): File "test_KVNet_LBA.py", line 532, in main() File "test_KVNet_LBA.py", line 386, in main ref_indx = ref_indx ) File "/home/compu/ylee/neuralrgbd/code/test_utils/test_KVNet.py", line 40, in test cam_intrinsics = Cam_Intrinsics, BV_predict = BV_predict) File "/home/compu/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, kwargs) File "/home/compu/.local/lib/python3.5/site-packages/torch/nn/parallel/data_parallel.py", line 141, in forward return self.module(*inputs[0], *kwargs[0]) File "/home/compu/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(input, kwargs) File "/home/compu/ylee/neuralrgbd/code/models/KVNET.py", line 124, in forward ref_frame, src_frames, src_cam_poses, BV_predict = None, debug_ipdb= False) File "/home/compu/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, kwargs) File "/home/compu/ylee/neuralrgbd/code/models/basic.py", line 245, in forward feat_imgs_layer_1, feat_imgs = self.feature_extraction(torch.cat((src_frames[0,...], ref_frame),dim=0)) File "/home/compu/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, *kwargs) File "/home/compu/ylee/neuralrgbd/code/models/basic.py", line 50, in forward feat_img_layer1, feat_img_final = self.feature_extraction(img) File "/home/compu/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(input, kwargs) File "/home/compu/ylee/neuralrgbd/code/models/psm_submodule.py", line 142, in forward output = self.firstconv(x) File "/home/compu/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, kwargs) File "/home/compu/.local/lib/python3.5/site-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/home/compu/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, *kwargs) File "/home/compu/.local/lib/python3.5/site-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/home/compu/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(input, kwargs) File "/home/compu/.local/lib/python3.5/site-packages/torch/nn/modules/conv.py", line 320, in forward self.padding, self.dilation, self.groups) RuntimeError: cuda runtime error (11) : invalid argument at /pytorch/aten/src/THC/THCGeneral.cpp:405

what could be the problem?

[cxlcl]

The outputs show that the DSO is not returning valid poses. Could you double check if the DSO is properly built ? BTW, we made some small changes to the DSO code and it's highly recommended to use the modified version (included in this repository).

[ylee1123]

I used very recent code in the repository so I don't think the problem is related to changes in DSO code. Also, I checked that I have bin/dso_dataset in my folder in third_party/dso/build so it doesn't seem to be a problem. If you think if the DSO is not properly built, how can I figure it out?

[cxlcl]

I've uploaded the DSO outputs here. You may double check by: (1) directly checking the result_dso.txt in the ./dso_res folder; or (2) in the local_test_LBA.sh script, commenting out the run_dso.py part and just run the depth estimation part, but using the uploaded DSO results. If it works then it should indicate that the discrepancy makes a difference.