acts-project / algebra-plugins

Mozilla Public License 2.0
3 stars 10 forks source link

Fix identity error in CUDA debug builds with Eigen #47

Closed stephenswat closed 2 years ago

stephenswat commented 2 years ago

This is a fix to a bug that @krasznaa alerted me of yesterday, where under some very strict set of circumstances, the Eigen tests would fail to run, giving an invalid memory access error. For this to happen, the code must be compiled in Debug build mode.

Anyway, the error results from the fact that the matrices in our Eigen transform3 class are all defaulted to the Identity() transformation. While this should be fine, it apparently does not work well with CUDA. Thus, the solution is to remove these defaults and explicitly set the matrices to identity matrices using setIdentity().

Without going deep into the Eigen3 code, I can only speculate why this bug happens. One semi-likely explanation might be that the identity matrices are stored in the .data or .bss sections of the executable in order to allow direct copies, and that this memory is not available on the GPU. However, I am not sure whether this is the case.