Questions about Matlab evaluation

[xmlyqing00]

Hi,

Thanks for providing detailed training and evaluation codes. In the last step of evaluations, running the BaseEval_web.m in Matlab to compute the score takes so long. About 1 hour for one scan. Is it normal?

Thanks

[ss2lyf]

Hi,

Thanks for providing detailed training and evaluation codes. In the last step of evaluations, running the BaseEval_web.m in Matlab to compute the score takes so long. About 1 hour for one scan. Is it normal?

Thanks What model is your computer's CPU? I use an i5 9400f, and each scene takes about thirty or forty minutes. If the memory is large enough, you can consider opening several matlab windows, or use matlab -nodesktop on the command line to test the results in parallel.

[FangjinhuaWang]

Hi,

Yes, the run-time is also a bit long on my machine, e.g. about 1 hour per scene.

[xmlyqing00]

Hi,

My CPU is Intel Xeon E5-2630. A little old but has multi-cores. I paste part of my running results here. You mentioned matlab -nodesktop can run the program in parallel. It seems what I am looking for. Could you tell me more details about it? Thanks.

cSet =
    77
DataInName =
    '/Ship03/Sources/MVS/PatchmatchNet/outputs/dtu_test/scan77/fused.ply'
EvalName =
    '/Ship03/Sources/MVS/PatchmatchNet/outputs/fusion/patchmatchnet_Eval_77.mat'
/Ship03/Sources/MVS/PatchmatchNet/outputs/dtu_test/scan77/fused.ply
ans =
     2    17
Elapsed time is 544.623314 seconds.
downsample factor: 4.4507
Elapsed time is 190.282952 seconds.
Computing Data 2 Stl distances
Elapsed time is 417.550732 seconds.
Computing Stl 2 Data distances
Distances computed
Elapsed time is 153.869475 seconds.
Saving results
Elapsed time is 154.281166 seconds.
Elapsed time is 168.462179 seconds.
ans =
     2    46
mean/median Data (acc.) 0.911920/0.374731
mean/median Stl (comp.) 0.344095/0.157903
cSet =
   110
DataInName =
    '/Ship03/Sources/MVS/PatchmatchNet/outputs/dtu_test/scan110/fused.ply'
EvalName =
    '/Ship03/Sources/MVS/PatchmatchNet/outputs/fusion/patchmatchnet_Eval_110.mat'
/Ship03/Sources/MVS/PatchmatchNet/outputs/dtu_test/scan110/fused.ply
ans =
     2    46
Elapsed time is 3110.646320 seconds.
downsample factor: 5.0153
Elapsed time is 1679.608172 seconds.
Computing Data 2 Stl distances
Elapsed time is 195.569941 seconds.
Computing Stl 2 Data distances
Distances computed
Elapsed time is 497.376512 seconds.
Saving results
Elapsed time is 498.258483 seconds.
Elapsed time is 523.000857 seconds.
ans =
     4    21
mean/median Data (acc.) 0.381386/0.235894
mean/median Stl (comp.) 0.235066/0.143738
cSet =
   114
DataInName =
    '/Ship03/Sources/MVS/PatchmatchNet/outputs/dtu_test/scan114/fused.ply'
EvalName =
    '/Ship03/Sources/MVS/PatchmatchNet/outputs/fusion/patchmatchnet_Eval_114.mat'
/Ship03/Sources/MVS/PatchmatchNet/outputs/dtu_test/scan114/fused.ply
ans =
     4    21
Elapsed time is 3695.510579 seconds.
downsample factor: 6.3373
Elapsed time is 2401.884661 seconds.
Computing Data 2 Stl distances
Elapsed time is 257.073296 seconds.
Computing Stl 2 Data distances
Distances computed
Elapsed time is 406.591941 seconds.
Saving results
Elapsed time is 408.003718 seconds.
Elapsed time is 433.095527 seconds.
ans =
     6    19
mean/median Data (acc.) 0.273182/0.210140
mean/median Stl (comp.) 0.209055/0.153379
cSet =
   118
DataInName =
    '/Ship03/Sources/MVS/PatchmatchNet/outputs/dtu_test/scan118/fused.ply'
EvalName =
    '/Ship03/Sources/MVS/PatchmatchNet/outputs/fusion/patchmatchnet_Eval_118.mat'
/Ship03/Sources/MVS/PatchmatchNet/outputs/dtu_test/scan118/fused.ply
ans =
     6    19
Elapsed time is 3140.886745 seconds.
downsample factor: 5.3363
Elapsed time is 2036.301528 seconds.
Computing Data 2 Stl distances
Elapsed time is 259.386764 seconds.
Computing Stl 2 Data distances
Distances computed
Elapsed time is 480.471163 seconds.
Saving results
Elapsed time is 481.900043 seconds.
Elapsed time is 513.897147 seconds.
ans =
     8     2
mean/median Data (acc.) 0.276457/0.211285
mean/median Stl (comp.) 0.297834/0.160435

[ss2lyf]

Hi,

Thanks for providing detailed training and evaluation codes. In the last step of evaluations, running the BaseEval_web.m in Matlab to compute the score takes so long. About 1 hour for one scan. Is it normal?

Thanks

Actually, what I mean is that you can make several copies of BaseEvalMain_web.m,and then first divide the test scene numbers in the UsedSets list into several subsets that do not intersect. Then no matter if you open multiple matlab windows or use the command line (if you are using a linux desktop environment, use matlab -nodesktop to run matlab on the command line without opening the gui interface). Next, you can run the BaseEvalMain_web.m scripts you copied earlier at the same time, each script is responsible for testing a part of the scenario. Because the test script will save the test results, after all the scripts are finished, run the BaseEvalMain_web.m that contains all the scene sequences to get the final result. But it should be noted that the test script will take up a lot of memory. For example, the physical memory and virtual memory of my computer are both 32 GB, and I can only run four test scripts at the same time.

[xmlyqing00]

I see. Thanks for your detailed explanation. It sounds great. I will try that.