Testing on DTU scan1 & 4

[karimcossentini]

I tested MSVformer on DTU scan 1 and 4 using weights provided on one drive, however it seems that the ply file generated as output is empty (does not contains 3D points)

---

filtering: 100% 49/49 [00:00<00:00, 59.34it/s] Convert mvsnet output to gipuma input convert depth maps: 0% 0/49 [00:00<?, ?it/s]1769472

convert depth maps: 2% 1/49 [00:00<00:16, 2.93it/s]1769472

convert depth maps: 4% 2/49 [00:00<00:17, 2.66it/s]1769472

convert depth maps: 6% 3/49 [00:01<00:17, 2.60it/s]1769472

convert depth maps: 8% 4/49 [00:01<00:17, 2.58it/s]1769472

convert depth maps: 10% 5/49 [00:01<00:17, 2.49it/s]1769472

convert depth maps: 12% 6/49 [00:02<00:17, 2.48it/s]1769472

convert depth maps: 14% 7/49 [00:02<00:17, 2.44it/s]1769472

convert depth maps: 16% 8/49 [00:03<00:17, 2.37it/s]1769472

convert depth maps: 18% 9/49 [00:03<00:16, 2.40it/s]1769472

convert depth maps: 20% 10/49 [00:04<00:17, 2.18it/s]1769472

convert depth maps: 22% 11/49 [00:04<00:17, 2.16it/s]1769472

convert depth maps: 24% 12/49 [00:05<00:16, 2.21it/s]1769472

convert depth maps: 27% 13/49 [00:05<00:15, 2.29it/s]1769472

convert depth maps: 29% 14/49 [00:05<00:14, 2.37it/s]1769472

convert depth maps: 31% 15/49 [00:06<00:14, 2.30it/s]1769472

convert depth maps: 33% 16/49 [00:06<00:14, 2.32it/s]1769472

convert depth maps: 35% 17/49 [00:07<00:13, 2.33it/s]1769472

convert depth maps: 37% 18/49 [00:07<00:12, 2.48it/s]1769472

convert depth maps: 39% 19/49 [00:07<00:11, 2.62it/s]1769472

convert depth maps: 41% 20/49 [00:08<00:10, 2.74it/s]1769472

convert depth maps: 43% 21/49 [00:08<00:09, 2.87it/s]1769472

convert depth maps: 45% 22/49 [00:08<00:09, 2.96it/s]1769472

convert depth maps: 47% 23/49 [00:09<00:08, 3.02it/s]1769472

convert depth maps: 49% 24/49 [00:09<00:08, 3.03it/s]1769472

convert depth maps: 51% 25/49 [00:09<00:07, 3.13it/s]1769472

convert depth maps: 53% 26/49 [00:10<00:07, 3.10it/s]1769472

convert depth maps: 55% 27/49 [00:10<00:07, 3.10it/s]1769472

convert depth maps: 57% 28/49 [00:10<00:06, 3.14it/s]1769472

convert depth maps: 59% 29/49 [00:11<00:06, 3.18it/s]1769472

convert depth maps: 61% 30/49 [00:11<00:06, 3.10it/s]1769472

convert depth maps: 63% 31/49 [00:11<00:05, 3.14it/s]1769472

convert depth maps: 65% 32/49 [00:11<00:05, 3.20it/s]1769472

convert depth maps: 67% 33/49 [00:12<00:05, 3.11it/s]1769472

convert depth maps: 69% 34/49 [00:12<00:04, 3.14it/s]1769472

convert depth maps: 71% 35/49 [00:12<00:04, 3.14it/s]1769472

convert depth maps: 73% 36/49 [00:13<00:04, 3.11it/s]1769472

convert depth maps: 76% 37/49 [00:13<00:04, 2.64it/s]1769472

convert depth maps: 78% 38/49 [00:14<00:03, 2.76it/s]1769472

convert depth maps: 80% 39/49 [00:14<00:03, 2.89it/s]1769472

convert depth maps: 82% 40/49 [00:14<00:02, 3.02it/s]1769472

convert depth maps: 84% 41/49 [00:15<00:02, 3.13it/s]1769472

convert depth maps: 86% 42/49 [00:15<00:02, 3.07it/s]1769472

convert depth maps: 88% 43/49 [00:15<00:01, 3.11it/s]1769472

convert depth maps: 90% 44/49 [00:15<00:01, 3.18it/s]1769472

convert depth maps: 92% 45/49 [00:16<00:01, 3.15it/s]1769472

convert depth maps: 94% 46/49 [00:16<00:00, 3.16it/s]1769472

convert depth maps: 96% 47/49 [00:16<00:00, 3.17it/s]1769472

convert depth maps: 98% 48/49 [00:17<00:00, 3.17it/s]1769472

convert depth maps: 100% 49/49 [00:18<00:00, 2.62it/s] Run depth map fusion & filter ./fusibile/fusibile -input_folder /content/MVSFormer/output/scan1/points_mvsnet/ -p_folder /content/MVSFormer/output/scan1/points_mvsnet/cams/ -images_folder /content/MVSFormer/output/scan1/points_mvsnet/images/ --depth_min=0.001 --depth_max=100000 --normal_thresh=360 --disp_thresh=0.1 --num_consistent=2.0 -color_processing Command-line parameter error: unknown option -input_folder input folder is /content/MVSFormer/output/scan1/points_mvsnet/ image folder is /content/MVSFormer/output/scan1/points_mvsnet/images/ p folder is /content/MVSFormer/output/scan1/points_mvsnet/cams/ pmvs folder is numImages is 49 img_filenames is 49 Device memory used: 2312.110107MB Device memory used: 2312.110107MB P folder is /content/MVSFormer/output/scan1/points_mvsnet/cams/ numCameras is 49 Camera size is 49 Accepted intersection angle of central rays is 10.000000 to 30.000000 degrees Selected views: 49 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, Reading normals and depth from disk Size consideredIds is 49 Reading normal 0 Reading disp 0 Reading normal 1 Reading disp 1 Reading normal 2 Reading disp 2 Reading normal 3 Reading disp 3 Reading normal 4 Reading disp 4 Reading normal 5 Reading disp 5 Reading normal 6 Reading disp 6 Reading normal 7 Reading disp 7 Reading normal 8 Reading disp 8 Reading normal 9 Reading disp 9 Reading normal 10 Reading disp 10 Reading normal 11 Reading disp 11 Reading normal 12 Reading disp 12 Reading normal 13 Reading disp 13 Reading normal 14 Reading disp 14 Reading normal 15 Reading disp 15 Reading normal 16 Reading disp 16 Reading normal 17 Reading disp 17 Reading normal 18 Reading disp 18 Reading normal 19 Reading disp 19 Reading normal 20 Reading disp 20 Reading normal 21 Reading disp 21 Reading normal 22 Reading disp 22 Reading normal 23 Reading disp 23 Reading normal 24 Reading disp 24 Reading normal 25 Reading disp 25 Reading normal 26 Reading disp 26 Reading normal 27 Reading disp 27 Reading normal 28 Reading disp 28 Reading normal 29 Reading disp 29 Reading normal 30 Reading disp 30 Reading normal 31 Reading disp 31 Reading normal 32 Reading disp 32 Reading normal 33 Reading disp 33 Reading normal 34 Reading disp 34 Reading normal 35 Reading disp 35 Reading normal 36 Reading disp 36 Reading normal 37 Reading disp 37 Reading normal 38 Reading disp 38 Reading normal 39 Reading disp 39 Reading normal 40 Reading disp 40 Reading normal 41 Reading disp 41 Reading normal 42 Reading disp 42 Reading normal 43 Reading disp 43 Reading normal 44 Reading disp 44 Reading normal 45 Reading disp 45 Reading normal 46 Reading disp 46 Reading normal 47 Reading disp 47 Reading normal 48 Reading disp 48 Resizing globalstate to 49 Run cuda Run gipuma Grid size initrand is grid: 48-36 block: 32-32 Device memory used: 5189.402832MB Number of iterations is 8 Blocksize is 15x15 Disparity threshold is 0.100000 Normal threshold is 6.283185 Number of consistent points is 2 Cam scale is 1.000000 Fusing points Processing camera 0 Found 0.00 million points Processing camera 1 Found 0.00 million points Processing camera 2 Found 0.00 million points Processing camera 3 Found 0.00 million points Processing camera 4 Found 0.00 million points Processing camera 5 Found 0.00 million points Processing camera 6 Found 0.00 million points Processing camera 7 Found 0.00 million points Processing camera 8 Found 0.00 million points Processing camera 9 Found 0.00 million points Processing camera 10 Found 0.00 million points Processing camera 11 Found 0.00 million points Processing camera 12 Found 0.00 million points Processing camera 13 Found 0.00 million points Processing camera 14 Found 0.00 million points Processing camera 15 Found 0.00 million points Processing camera 16 Found 0.00 million points Processing camera 17 Found 0.00 million points Processing camera 18 Found 0.00 million points Processing camera 19 Found 0.00 million points Processing camera 20 Found 0.00 million points Processing camera 21 Found 0.00 million points Processing camera 22 Found 0.00 million points Processing camera 23 Found 0.00 million points Processing camera 24 Found 0.00 million points Processing camera 25 Found 0.00 million points Processing camera 26 Found 0.00 million points Processing camera 27 Found 0.00 million points Processing camera 28 Found 0.00 million points Processing camera 29 Found 0.00 million points Processing camera 30 Found 0.00 million points Processing camera 31 Found 0.00 million points Processing camera 32 Found 0.00 million points Processing camera 33 Found 0.00 million points Processing camera 34 Found 0.00 million points Processing camera 35 Found 0.00 million points Processing camera 36 Found 0.00 million points Processing camera 37 Found 0.00 million points Processing camera 38 Found 0.00 million points Processing camera 39 Found 0.00 million points Processing camera 40 Found 0.00 million points Processing camera 41 Found 0.00 million points Processing camera 42 Found 0.00 million points Processing camera 43 Found 0.00 million points Processing camera 44 Found 0.00 million points Processing camera 45 Found 0.00 million points Processing camera 46 Found 0.00 million points Processing camera 47 Found 0.00 million points Processing camera 48 Found 0.00 million points ELAPSED 1.522794 seconds Error: no kernel image is available for execution on the device Writing ply file /content/MVSFormer/output/scan1/points_mvsnet//consistencyCheck-20230529-221442//final3d_model.ply store 3D points to ply file

[ewrfcas]

I suggest checking whether the depth is correct. If all predicted depth maps are correct, the problem should be caused by the compiling of gipuma. See README for more details.

Tips: You should revise CUDA_NVCC_FLAGS in CMakeLists.txt according the gpu device you used. We set -gencode arch=compute_70,code=sm_70 instead of -gencode arch=compute_60,code=sm_60 with V100 GPUs. For other GPU types, you can follow... (Please refer to the README)

[karimcossentini]

In fact, I am using ground truth depth maps from Depths_raw zip file, so they should be correct, but In case I am using predicted ones how could I check if it's correct ? and as for the CUDA_NVCC_FLAGS in CMakeLists.txt I am using tesla T4 so I set -gencode arch=compute_75,code=sm_75 and still generating an empty ply file, I am investing quite an effort and time on it to make it up and running but have not figured out a workaround for it yet. Any suggestions ? thanks The logs are down below:

model mvsnet <class 'str'>
device None <class 'NoneType'>
config None <class 'NoneType'>
dataset dtu <class 'str'>
testpath /content/MVSFormer/dtu <class 'str'>
testpath_single_scene None <class 'NoneType'>
testlist /content/MVSFormer/lists/dtu/test.txt <class 'str'>
exp_name None <class 'NoneType'>
batch_size 1 <class 'int'>
numdepth 192 <class 'int'>
resume /content/MVSFormer/pretrained_weights/mvs_best.pth <class 'str'>
outdir /content/MVSFormer/output <class 'str'>
display False <class 'bool'>
share_cr False <class 'bool'>
ndepths None <class 'NoneType'>
depth_interals_ratio None <class 'NoneType'>
cr_base_chs 8,8,8 <class 'str'>
grad_method detach <class 'str'>
no_refinement False <class 'bool'>
full_res False <class 'bool'>
interval_scale 1.06 <class 'float'>
num_view 5 <class 'int'>
max_h 1152 <class 'int'>
max_w 1536 <class 'int'>
fix_res False <class 'bool'>
depth_scale 1.0 <class 'float'>
temperature 0.01 <class 'float'>
num_worker 4 <class 'int'>
save_freq 20 <class 'int'>
filter_method gipuma <class 'str'>
conf 0.1,0.1,0.1,0.1 <class 'str'>
thres_view 3 <class 'int'>
thres_disp 1.0 <class 'float'>
downsample None <class 'NoneType'>
dist_base 4.0 <class 'float'>
rel_diff_base 1300.0 <class 'float'>
fusibile_exe_path ./fusibile/fusibile <class 'str'>
prob_threshold 0.5,0.5,0.5,0.5 <class 'str'>
disp_threshold 0.1 <class 'float'>
num_consistent 2.0 <class 'float'>
use_short_range False <class 'bool'>
combine_conf True <class 'bool'>
tmp 1.0 <class 'float'>
tmps 5.0,5.0,5.0,1.0 <class 'str'>
save_all_confs False <class 'bool'>
######################################################################## ***Interval_Scale** 1.06 dataset test metas: 49 interval_scale:{'scan4': 1.06} /usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py:478: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary. warnings.warn(_create_warning_msg( drop_path_rate: --- 0.2 !!!No weight in ./pretrained_weights/alt_gvt_small.pth testing should neglect this. Loading checkpoint: /content/MVSFormer/pretrained_weights/mvs_best.pth ... /usr/local/lib/python3.8/dist-packages/torch/nn/functional.py:3487: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead. warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.") Iter 0/49, Time:3.415274143218994 Res:(1152, 1536) Iter 1/49, Time:1.397202491760254 Res:(1152, 1536) Iter 2/49, Time:1.409536600112915 Res:(1152, 1536) Iter 3/49, Time:1.4168994426727295 Res:(1152, 1536) Iter 4/49, Time:1.4076523780822754 Res:(1152, 1536) Iter 5/49, Time:1.4444632530212402 Res:(1152, 1536) Iter 6/49, Time:1.402644157409668 Res:(1152, 1536) Iter 7/49, Time:1.5373404026031494 Res:(1152, 1536) Iter 8/49, Time:1.377190113067627 Res:(1152, 1536) Iter 9/49, Time:1.3741869926452637 Res:(1152, 1536) Iter 10/49, Time:1.3821210861206055 Res:(1152, 1536) Iter 11/49, Time:1.4419941902160645 Res:(1152, 1536) Iter 12/49, Time:1.4461524486541748 Res:(1152, 1536) Iter 13/49, Time:1.4321582317352295 Res:(1152, 1536) Iter 14/49, Time:1.398536205291748 Res:(1152, 1536) Iter 15/49, Time:1.3773183822631836 Res:(1152, 1536) Iter 16/49, Time:1.3878734111785889 Res:(1152, 1536) Iter 17/49, Time:1.3845915794372559 Res:(1152, 1536) Iter 18/49, Time:1.3941154479980469 Res:(1152, 1536) Iter 19/49, Time:1.3992080688476562 Res:(1152, 1536) Iter 20/49, Time:1.4185020923614502 Res:(1152, 1536) Iter 21/49, Time:1.4437236785888672 Res:(1152, 1536) Iter 22/49, Time:1.473573923110962 Res:(1152, 1536) Iter 23/49, Time:1.4169540405273438 Res:(1152, 1536) Iter 24/49, Time:1.416062355041504 Res:(1152, 1536) Iter 25/49, Time:1.4074761867523193 Res:(1152, 1536) Iter 26/49, Time:1.4010851383209229 Res:(1152, 1536) Iter 27/49, Time:1.4064266681671143 Res:(1152, 1536) Iter 28/49, Time:1.39436674118042 Res:(1152, 1536) Iter 29/49, Time:1.403296947479248 Res:(1152, 1536) Iter 30/49, Time:1.4358813762664795 Res:(1152, 1536) Iter 31/49, Time:1.4404237270355225 Res:(1152, 1536) Iter 32/49, Time:1.4343783855438232 Res:(1152, 1536) Iter 33/49, Time:1.4601478576660156 Res:(1152, 1536) Iter 34/49, Time:1.477217197418213 Res:(1152, 1536) Iter 35/49, Time:1.5847489833831787 Res:(1152, 1536) Iter 36/49, Time:1.410426139831543 Res:(1152, 1536) Iter 37/49, Time:1.4033558368682861 Res:(1152, 1536) Iter 38/49, Time:1.4120643138885498 Res:(1152, 1536) Iter 39/49, Time:1.4281516075134277 Res:(1152, 1536) Iter 40/49, Time:1.4707086086273193 Res:(1152, 1536) Iter 41/49, Time:1.4015827178955078 Res:(1152, 1536) Iter 42/49, Time:1.3987183570861816 Res:(1152, 1536) Iter 43/49, Time:1.4093499183654785 Res:(1152, 1536) Iter 44/49, Time:1.4071369171142578 Res:(1152, 1536) Iter 45/49, Time:1.399526596069336 Res:(1152, 1536) Iter 46/49, Time:1.4065728187561035 Res:(1152, 1536) Iter 47/49, Time:1.4020535945892334 Res:(1152, 1536) Iter 48/49, Time:1.4119088649749756 Res:(1152, 1536) average time: 1.4612710329951073 filter depth map with probability map filtering: 100% 49/49 [00:01<00:00, 45.61it/s] Convert mvsnet output to gipuma input convert depth maps: 100% 49/49 [00:16<00:00, 2.91it/s] Run depth map fusion & filter ./fusibile/fusibile -input_folder /content/MVSFormer/output/scan4/points_mvsnet/ -p_folder /content/MVSFormer/output/scan4/points_mvsnet/cams/ -images_folder /content/MVSFormer/output/scan4/points_mvsnet/images/ --depth_min=0.001 --depth_max=100000 --normal_thresh=360 --disp_thresh=0.1 --num_consistent=2.0 -color_processing Command-line parameter error: unknown option -input_folder input folder is /content/MVSFormer/output/scan4/points_mvsnet/ image folder is /content/MVSFormer/output/scan4/points_mvsnet/images/ p folder is /content/MVSFormer/output/scan4/points_mvsnet/cams/ pmvs folder is numImages is 49 img_filenames is 49 Device memory used: 2079.326172MB Device memory used: 2079.326172MB P folder is /content/MVSFormer/output/scan4/points_mvsnet/cams/ numCameras is 49 Camera size is 49 Accepted intersection angle of central rays is 10.000000 to 30.000000 degrees Selected views: 49 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, Reading normals and depth from disk Size consideredIds is 49 Reading normal 0 Reading disp 0 Reading normal 1 Reading disp 1 Reading normal 2 Reading disp 2 Reading normal 3 Reading disp 3 Reading normal 4 Reading disp 4 Reading normal 5 Reading disp 5 Reading normal 6 Reading disp 6 Reading normal 7 Reading disp 7 Reading normal 8 Reading disp 8 Reading normal 9 Reading disp 9 Reading normal 10 Reading disp 10 Reading normal 11 Reading disp 11 Reading normal 12 Reading disp 12 Reading normal 13 Reading disp 13 Reading normal 14 Reading disp 14 Reading normal 15 Reading disp 15 Reading normal 16 Reading disp 16 Reading normal 17 Reading disp 17 Reading normal 18 Reading disp 18 Reading normal 19 Reading disp 19 Reading normal 20 Reading disp 20 Reading normal 21 Reading disp 21 Reading normal 22 Reading disp 22 Reading normal 23 Reading disp 23 Reading normal 24 Reading disp 24 Reading normal 25 Reading disp 25 Reading normal 26 Reading disp 26 Reading normal 27 Reading disp 27 Reading normal 28 Reading disp 28 Reading normal 29 Reading disp 29 Reading normal 30 Reading disp 30 Reading normal 31 Reading disp 31 Reading normal 32 Reading disp 32 Reading normal 33 Reading disp 33 Reading normal 34 Reading disp 34 Reading normal 35 Reading disp 35 Reading normal 36 Reading disp 36 Reading normal 37 Reading disp 37 Reading normal 38 Reading disp 38 Reading normal 39 Reading disp 39 Reading normal 40 Reading disp 40 Reading normal 41 Reading disp 41 Reading normal 42 Reading disp 42 Reading normal 43 Reading disp 43 Reading normal 44 Reading disp 44 Reading normal 45 Reading disp 45 Reading normal 46 Reading disp 46 Reading normal 47 Reading disp 47 Reading normal 48 Reading disp 48 Resizing globalstate to 49 Run cuda Run gipuma Grid size initrand is grid: 48-36 block: 32-32 Device memory used: 4956.618652MB Number of iterations is 8 Blocksize is 15x15 Disparity threshold is 0.100000 Normal threshold is 6.283185 Number of consistent points is 2 Cam scale is 1.000000 Fusing points Processing camera 0 Found 0.00 million points Processing camera 1 Found 0.00 million points Processing camera 2 Found 0.00 million points Processing camera 3 Found 0.00 million points Processing camera 4 Found 0.00 million points Processing camera 5 Found 0.00 million points Processing camera 6 Found 0.00 million points Processing camera 7 Found 0.00 million points Processing camera 8 Found 0.00 million points Processing camera 9 Found 0.00 million points Processing camera 10 Found 0.00 million points Processing camera 11 Found 0.00 million points Processing camera 12 Found 0.00 million points Processing camera 13 Found 0.00 million points Processing camera 14 Found 0.00 million points Processing camera 15 Found 0.00 million points Processing camera 16 Found 0.00 million points Processing camera 17 Found 0.00 million points Processing camera 18 Found 0.00 million points Processing camera 19 Found 0.00 million points Processing camera 20 Found 0.00 million points Processing camera 21 Found 0.00 million points Processing camera 22 Found 0.00 million points Processing camera 23 Found 0.00 million points Processing camera 24 Found 0.00 million points Processing camera 25 Found 0.00 million points Processing camera 26 Found 0.00 million points Processing camera 27 Found 0.00 million points Processing camera 28 Found 0.00 million points Processing camera 29 Found 0.00 million points Processing camera 30 Found 0.00 million points Processing camera 31 Found 0.00 million points Processing camera 32 Found 0.00 million points Processing camera 33 Found 0.00 million points Processing camera 34 Found 0.00 million points Processing camera 35 Found 0.00 million points Processing camera 36 Found 0.00 million points Processing camera 37 Found 0.00 million points Processing camera 38 Found 0.00 million points Processing camera 39 Found 0.00 million points Processing camera 40 Found 0.00 million points Processing camera 41 Found 0.00 million points Processing camera 42 Found 0.00 million points Processing camera 43 Found 0.00 million points Processing camera 44 Found 0.00 million points Processing camera 45 Found 0.00 million points Processing camera 46 Found 0.00 million points Processing camera 47 Found 0.00 million points Processing camera 48 Found 0.00 million points ELAPSED 1.583567 seconds Error: no kernel image is available for execution on the device Writing ply file /content/MVSFormer/output/scan4/points_mvsnet//consistencyCheck-20230530-093109//final3d_model.ply store 3D points to ply file

[ewrfcas]

This issue should be addressed.

[z111aa]

I have the same problem. How can I solve it

[wscstrive]

The cuda version is inconsistent with the fusibile version

[ewrfcas]

So, we need to re-compile the fusible under the current cuda environment.