search

DGtal-team / DGtal

Digital Geometry Tools and Algorithm Library
https://dgtal.org
GNU Lesser General Public License v3.0
370 stars 115 forks source link

Performance regression on testVoxelComplex since #1369 #1393

Open rolanddenis opened 5 years ago

rolanddenis commented 5 years ago

It's a bit confusing but the time needed by testVoxelComplex seems to have increased since the PR #1369 :sweat_smile:

With GNU 5.5 and Boost 1.63 in Release mode, it increases from 9.4s to 14.6s:

$ git checkout 1d87cee245a6a017d6166d4061c28e1defbcd5fa
HEAD is now at 1d87cee... ENH: Remove Object from VoxelComplex
$ make testVoxelComplex
...
$ time ./tests/topology/testVoxelComplex 2> /dev/null
===============================================================================
All tests passed (88 assertions in 18 test cases)

real    0m14.629s
user    0m14.040s
sys 0m0.576s

versus

$ git checkout 1d87cee245a6a017d6166d4061c28e1defbcd5fa^1
Previous HEAD position was 1d87cee... ENH: Remove Object from VoxelComplex
HEAD is now at 60e0e2b... Cleaning up slack notifications
$ make testVoxelComplex
...
$ time ./tests/topology/testVoxelComplex 2> /dev/null
===============================================================================
All tests passed (96 assertions in 19 test cases)

real    0m9.383s
user    0m8.844s
sys 0m0.532s

and in Debug mode, it increases from 1m13s to 3m51s !

It is also visible in Travis jobs between https://travis-ci.org/DGtal-team/DGtal/jobs/473287981#L2525 (98.13s) and https://travis-ci.org/DGtal-team/DGtal/jobs/475986541#L2560 (251.25s).

dcoeurjo commented 5 years ago

Thanks for the report... Ping @phcerdan & @JacquesOlivierLachaud ?

phcerdan commented 5 years ago

Thanks for the test, and the ping. I will have a look!

phcerdan commented 5 years ago

Hey @rolanddenis, thanks for the tests and the report!

The time regression only happens when DGtal build is on Debug mode... The tests in Travis are not run in Release mode, as shown in the CMake configuration output here: https://travis-ci.org/DGtal-team/DGtal/jobs/473287981#L1566

With CMAKE_BUILD_TYPE:STRING=RelWithDebInfo or Release, the ctest -R testVoxelComplex -V is slightly faster in all the checks. The speed-up is even more significant when running it in bigger images.

In debug mode I can reproduce the regression... but it is insignificant (from 109s to 112s)

Could you double check that you were compiling DGtal in Release mode in your local test?

Maybe I introduced a debug check there that is consuming too many resources, but I cannot really reproduce it as clear as you.

For context, the speed up difference is from ~5 seconds in RelWithDebInfo to ~110 seconds in Debug mode.