As the number of benchmarks integrated into SLAMBench grows, it becomes increasingly hard to identify versions of the dependencies that would work across the board for all the benchmarks.
Issues such as #32 (related: https://github.com/ethz-asl/okvis/issues/75), as well as having had issues with ORB-SLAM2 producing incorrect results due to using a system-installed version of g2o highlight the importance of this. In addition, while trying to integrate the branches in SLAMBench 3.0, I haven't been able to find a single OpenCV version that works across all benchmarks. I think it's worth looking into what the options are for this - we can only expect that new benchmarks will be integrated, with up-to-date dependencies, hence producing more conflicts with the old benchmarks.
One option is to have each benchmark pull in its own dependencies. There are important downsides to this: large dependencies such as OpenCV take a long time to download and compile, not to mention that they take up lots of space. Having this replicated over and over for each benchmark does not seem ideal. Moreover, when performing head-to-head comparison, it would be desirable to use the same versions of common dependencies, to avoid the possibility that the results are influenced by dependency versions.
@bbodin
As the number of benchmarks integrated into SLAMBench grows, it becomes increasingly hard to identify versions of the dependencies that would work across the board for all the benchmarks.
Issues such as #32 (related: https://github.com/ethz-asl/okvis/issues/75), as well as having had issues with ORB-SLAM2 producing incorrect results due to using a system-installed version of g2o highlight the importance of this. In addition, while trying to integrate the branches in SLAMBench 3.0, I haven't been able to find a single OpenCV version that works across all benchmarks. I think it's worth looking into what the options are for this - we can only expect that new benchmarks will be integrated, with up-to-date dependencies, hence producing more conflicts with the old benchmarks.
One option is to have each benchmark pull in its own dependencies. There are important downsides to this: large dependencies such as OpenCV take a long time to download and compile, not to mention that they take up lots of space. Having this replicated over and over for each benchmark does not seem ideal. Moreover, when performing head-to-head comparison, it would be desirable to use the same versions of common dependencies, to avoid the possibility that the results are influenced by dependency versions.