search

BlueBrain / HighFive

HighFive - Header-only C++ HDF5 interface
https://bluebrain.github.io/HighFive/
Boost Software License 1.0
675 stars 160 forks source link

Eigen integration #302

Closed lletournel closed 4 years ago

lletournel commented 4 years ago

Hi, I am currently trying HighFive, in order to use it, concurrently with Eigen, in my project (FRyDoM : https://frydom.org/). The code is open-source but you need to register to access the repo (no need to for this issue).

I am having some difficulties using HighFive with Eigen. I created a dummy project, with the following CMakeLists

cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(foo)
set(CMAKE_CXX_STANDARD 14)
include(FetchContent)
FetchContent_Declare(highfive
            GIT_REPOSITORY https://github.com/BlueBrain/HighFive.git
            GIT_TAG master
            )
FetchContent_GetProperties(highfive)
FetchContent_Populate(highfive)
set(HIGHFIVE_USE_EIGEN ON CACHE BOOL "" FORCE)
add_subdirectory(${highfive_SOURCE_DIR} ${highfive_BINARY_DIR})
find_package(PkgConfig)
pkg_check_modules(EIGEN3 REQUIRED eigen3)
include_directories(SYSTEM ${EIGEN3_INCLUDE_DIRS})
add_executable(example_Eigen example_Eigen.cpp)
target_link_libraries(example_Eigen HighFive)

and example_Eigen.cpp contains

#include <iostream>
#include <highfive/H5Easy.hpp>
#include <Eigen/Eigen>

int main()
{
  {
    Eigen::MatrixXd D = Eigen::MatrixXd::Random(10,5);
    H5Easy::File file("example.h5", H5Easy::File::Overwrite);

    H5Easy::dump(file, "/path/to/D", D);

    auto a = H5Easy::load<Eigen::MatrixXd>(file, "/path/to/D");
    std::cout << a << std::endl;
  }
  return 0;
}

I got a compile error

[ 50%] Building CXX object CMakeFiles/example_Eigen.dir/example_Eigen.cpp.o
In file included from /home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/H5DataSpace.hpp:158,
                 from /home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/H5Attribute.hpp:14,
                 from /home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/H5File.hpp:75,
                 from /home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/H5Easy.hpp:38,
                 from /home/lletourn/Documents/DEV/Test/test_HighFive/example_Eigen.cpp:2:
/home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/bits/H5Dataspace_misc.hpp: In instantiation of ‘static HighFive::DataSpace HighFive::DataSpace::From(const ScalarValue&) [with ScalarValue = Eigen::Matrix<double, -1, -1>]’:
/home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/h5easy_bits/H5Easy_scalar.hpp:28:70:   required from ‘static HighFive::DataSet H5Easy::detail::scalar::dump_impl<T, E>::run(HighFive::File&, const string&, const T&) [with T = Eigen::Matrix<double, -1, -1>; E = void; std::__cxx11::string = std::__cxx11::basic_string<char>]’
/home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/h5easy_bits/H5Easy_scalar.hpp:137:49:   required from ‘HighFive::DataSet H5Easy::dump(HighFive::File&, const string&, const T&, H5Easy::DumpMode) [with T = Eigen::Matrix<double, -1, -1>; std::__cxx11::string = std::__cxx11::basic_string<char>]’
/home/lletourn/Documents/DEV/Test/test_HighFive/example_Eigen.cpp:11:39:   required from here
/home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/bits/H5Dataspace_misc.hpp:136:43: error: static assertion failed: Only the following types are supported by DataSpace::From: 
  signed_arithmetic_types = int | long | float | double 
  unsigned_arithmetic_types = unsigned signed_arithmetic_types 
  string_types = std::string 
  all_basic_types = string_types | unsigned_arithmetic_types | signed_arithmetic_types 
   stl_container_types = std::vector<all_basic_types>   boost_container_types = boost::numeric::ublas::matrix<all_basic_types> | boost::multi_array<all_basic_types> 
  all_supported_types = all_basic_types | stl_container_types | boost_container_types  eigen_matrix_type = Eigen::Matrix<signed_arithmetic_type> | Eigen::VectorXX
         (std::is_arithmetic<ScalarValue>::value ||
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          std::is_enum<ScalarValue>::value ||
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
          std::is_same<std::string, ScalarValue>::value),
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/bits/H5Attribute_misc.hpp:25,
                 from /home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/H5Attribute.hpp:75,
                 from /home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/H5File.hpp:75,
                 from /home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/H5Easy.hpp:38,
                 from /home/lletourn/Documents/DEV/Test/test_HighFive/example_Eigen.cpp:2:
/home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/bits/H5Converter_misc.hpp: In instantiation of ‘HighFive::details::data_converter<Scalar, Enable>::data_converter(const Scalar&, const HighFive::DataSpace&) [with Scalar = Eigen::Matrix<double, -1, -1>; Enable = void]’:
/home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/bits/../bits/H5Slice_traits_misc.hpp:238:44:   required from ‘void HighFive::SliceTraits< <template-parameter-1-1> >::write(const T&) [with T = Eigen::Matrix<double, -1, -1>; Derivate = HighFive::DataSet]’
/home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/h5easy_bits/H5Easy_scalar.hpp:29:9:   required from ‘static HighFive::DataSet H5Easy::detail::scalar::dump_impl<T, E>::run(HighFive::File&, const string&, const T&) [with T = Eigen::Matrix<double, -1, -1>; E = void; std::__cxx11::string = std::__cxx11::basic_string<char>]’
/home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/h5easy_bits/H5Easy_scalar.hpp:137:49:   required from ‘HighFive::DataSet H5Easy::dump(HighFive::File&, const string&, const T&, H5Easy::DumpMode) [with T = Eigen::Matrix<double, -1, -1>; std::__cxx11::string = std::__cxx11::basic_string<char>]’
/home/lletourn/Documents/DEV/Test/test_HighFive/example_Eigen.cpp:11:39:   required from here
/home/lletourn/Documents/DEV/Test/test_HighFive/cmake-build-debug/_deps/highfive-src/include/highfive/bits/H5Converter_misc.hpp:121:52: error: static assertion failed: supported datatype should be an arithmetic value, a std::string or a container/array
         static_assert((std::is_arithmetic<Scalar>::value ||
                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                        std::is_enum<Scalar>::value ||
                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
                        std::is_same<std::string, Scalar>::value),
                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CMakeFiles/example_Eigen.dir/build.make:62: recipe for target 'CMakeFiles/example_Eigen.dir/example_Eigen.cpp.o' failed
make[3]: *** [CMakeFiles/example_Eigen.dir/example_Eigen.cpp.o] Error 1
CMakeFiles/Makefile2:72: recipe for target 'CMakeFiles/example_Eigen.dir/all' failed
make[2]: *** [CMakeFiles/example_Eigen.dir/all] Error 2
CMakeFiles/Makefile2:84: recipe for target 'CMakeFiles/example_Eigen.dir/rule' failed
make[1]: *** [CMakeFiles/example_Eigen.dir/rule] Error 2
Makefile:164: recipe for target 'example_Eigen' failed
make: *** [example_Eigen] Error 2

However the easy_load_dump.cpp compiles and runs correctly. The D dataset with the Eigen matrix is in the example.h5. I had to specify set(CMAKE_CXX_STANDARD 14) as in #269, though.

tdegeus commented 4 years ago

I can reproduce your error. The problem is that apparently the define H5_USE_EIGEN is not defined by the CMake target. The work-around is simple: include Eigen before HighFive. HighFive will detect that Eigen is included and enable its Eigen features.

To facilitate CMake debugging let me report that I get the same error (caused by disabled Eigen features) with:

project(test)

cmake_minimum_required(VERSION 3.16)

set(HIGHFIVE_USE_EIGEN 1)
find_package(HighFive)
find_package(PkgConfig)
pkg_check_modules(EIGEN3 REQUIRED eigen3)
include_directories(SYSTEM ${EIGEN3_INCLUDE_DIRS})
add_executable(test main.cpp)
target_link_libraries(test PRIVATE HighFive)

While I can a failure on the include of Eigen (outside HighFive) when I use 

project(test)

cmake_minimum_required(VERSION 3.16)

set(HIGHFIVE_USE_EIGEN 1)
find_package(HighFive)
add_executable(test main.cpp)
target_link_libraries(test PRIVATE HighFive)

(which should be equally legal)

lletournel commented 4 years ago

Thank you for the work-around solution, it works indeed. Please let me know if/when the underlying problem is fixed.

I would like to know also if it is possible to specify the path to eigen sources to HighFive. We have a lot of dependencies in FRyDoM, that we manage through FetchContent, including eigen. (The eigen we use in FRyDoM is not the one which might be installed on the system). So the includes "Eigen/Eigen" in HighFive are not currently working...

tdegeus commented 4 years ago

If you set(HIGHFIVE_USE_EIGEN 0) before loading HighFive and you load Eigen yourself you should be good to go. Be sure to either #include <Eigen/Eigen> before #include <highfive/H5Easy.hpp> or #define H5_USE_EIGEN before #include <highfive/H5Easy.hpp>

lletournel commented 4 years ago

Well it was more an include problem, but since HighFive is header only, I only included the highfive/include directory along with the HDF5 and Eigen include directories to make it work.

I managed to load and write Eigen::Vector3d data with H5Easy but not with H5 : I still got the "static assertion failed" compile error. It is related to #270 and not ready yet?

tdegeus commented 4 years ago

All I can really say is that H5Easy's Eigen support is for the moment independent of the Eigen support deeper down in the library. In H5Easy there are several Eigen overloads to make sure that your compiler finds a satisfying match. Maybe the others can comment how this is dealt with deeper down in the library, and if this will be improved in the future.

lletournel commented 4 years ago

Ok, thank you very much for the quick answer and all the great job done in HighFive ! I'll go with H5Easy for now and will keep an eye on HighFive future developments.

alexsavulescu commented 4 years ago

Hello @lletournel, we are planning on doing more work for Eigen but we haven't set up a timeline for it yet, H5Easy is a sound choice at the moment.

Could you have a look here, give it a try according to your use case and give us some feedback : https://github.com/BlueBrain/HighFive/blob/master/tests/unit/tests_high_five_base.cpp#L1534

Thank you

ferdonline commented 4 years ago

@lletournel This issue got automatically closed since I mentioned it as related to the last PR. Could you please verify if indeed the issue no longer exists with the latest master, or if it is still there? Either way, as mentioned by @alexsavulescu we should further rework Eigen integration.

lletournel commented 4 years ago

Hi Fernando,

I am sorry, I have little to no time anymore to spend on helping you on this issue. As far as I tested with the integration of HighFive in my project, I still got the static assertion failed error, even with the latest master. But since I don't rely on your CMake targets, it may not be surprising... I couldn't give a try also to tests_high_five_base.cpp.

I will come back to this in a few months and, from what I saw, HighFive might change quite a lot ! I will keep you updated then !

Le 02/03/2020 à 22:47, Fernando Pereira a écrit :

@lletournel https://github.com/lletournel This issue got automatically closed since I mentioned it as related to the last PR. Could you please verify if indeed the issue no longer exists with the latest master, or if it is still there? Either way, as mentioned by @alexsavulescu https://github.com/alexsavulescu we should further rework Eigen integration.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BlueBrain/HighFive/issues/302?email_source=notifications&email_token=AOV4LOZY4SUYLCHOQAPNWKLRFQSO3A5CNFSM4K5RD5OKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENRDU3Y#issuecomment-593640047, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOV4LOZORZYFCBUG57PFOSDRFQSO3ANCNFSM4K5RD5OA.