search

foonathan / memory

STL compatible C++ memory allocator library using a new RawAllocator concept that is similar to an Allocator but easier to use and write.
https://memory.foonathan.net
zlib License
1.5k stars 196 forks source link

cross compiling for QNX segfaults #72

Closed briansoe66 closed 4 years ago

briansoe66 commented 4 years ago

When cross compile master on QNX, I get a segmentation fault from container_node_sizes_impl.hpp

CMake command with toolchain: qnx_toolchain.zip

cmake .. -DCMAKE_TOOLCHAIN_FILE=~/Downloads/qnx_toolchain.cmake -DCMAKE_INSTALL_PREFIX=/home/brian/qnx700/target/qnx7/x86/usr/local -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DFOONATHAN_MEMORY_BUILD_EXAMPLES=OFF -DFOONATHAN_MEMORY_BUILD_TESTS=OFF -DFOONATHAN_MEMORY_BUILD_TOOLS=ON

Output: [ 13%] Generating container_node_sizes_impl.hpp cd /home/brian/Downloads/foonathan_memory/build/src && ../tool/nodesize_dbg --code --alignof "FOONATHAN_ALIGNOF(T)" /home/brian/Downloads/foonathan_memory/build/src/container_node_sizes_impl.hpp Segmentation fault (core dumped) src/CMakeFiles/foonathan_memory.dir/build.make:64: recipe for target 'src/container_node_sizes_impl.hpp' failed make[2]: *** [src/container_node_sizes_impl.hpp] Error 139 make[2]: Leaving directory '/home/brian/Downloads/foonathan_memory/build' CMakeFiles/Makefile2:88: recipe for target 'src/CMakeFiles/foonathan_memory.dir/all' failed make[1]: *** [src/CMakeFiles/foonathan_memory.dir/all] Error 2 make[1]: Leaving directory '/home/brian/Downloads/foonathan_memory/build' Makefile:132: recipe for target 'all' failed make: *** [all] Error 2 build_output.txt

MiguelCompany commented 4 years ago

Looking at your toolchain file, it seems that you are building for an x86 system. If you are cross compiling from an x64 system, you have two options, I think.

  1. Allowing your x64 system to run 32bit applications. Some information here
  2. Using an emulator, by adding set(CMAKE_CROSSCOMPILING_EMULATOR /path/to/emulator) to your toolchain
foonathan commented 4 years ago

This doesn't seem to be an issue in memory itself.

cjdesno commented 4 years ago

in case anybody else stumbles across this issue in future with the same problem:

let's look at the build as a 3-step process.

  1. it first builds tool/nodesize_dbg,
  2. then runs nodesize_dbg to determine some properties of the system and generate the include/foonathan/memory/detail/container_node_sizes_impl.hpp headerfile,
  3. and finally compiles the rest of the library now that all headers are present.

when cross-compiling, step 2 will fail, as it is attempting to run the nodesize_dbg binary (built for the target system) on the host system rather than the target.

the simplest workaround for this is the following:

  1. run the build (it will fail, as seen at the top of this thread)
  2. when the build fails, copy the nodesize_dbg binary from the host's build directory to the target, then run it with the appropriate arguments to generate the necessary header file (you can find the correct invocation in the build log output or the cmake files)
  3. copy the generated headerfile back from the target to the correct location in the project directory on the host
  4. comment out the cmake line that runs nodesize_dbg
  5. rerun the build

(i'm sure there's a cleaner way to do it, but this approach works well enough for a one-off build)

(p.s.: if you instead changed up the cmake files to build nodesize_dbg for the host, allowing it to run successfully, the information it would gather from the host system is likely significantly different from the target environment. this would cause the build to succeed but likely lead to runtime errors in any applications linked against the library, so we don't want to do this.)

razr commented 4 years ago

Hello @foonathan, I have the same issue with the VxWorks cross-compilation as well :) I think it could be handled a little bit more gently, e.g. if CMAKE_CROSSCOMPILING_EMULATOR is not defined - don't run it at all, it will fail anyway, and also probably check if container_node_sizes_impl.hpp is already present, don't run this check as @cjdesno has suggested? I'm not sure whether it makes sense to provide you e.g. with an autogenerated version for VxWorks. If you are ready to accept it, I'd be happy to do so. Kind Regards, Andrei

cjdesno commented 4 years ago

@razr oh, to be clear, i have no suggestion for a better way the project build could handle this case; my comment was just for a user workaround to this problem. for example, one issue with changing the project code to not generate the header if it already exists, is that when the header-generation code is updated, an incremental build would fail to properly regenerate a new header. so i think it would be difficult to streamline it more than it already is, so long as this header generation stage remains part of the build.

foonathan commented 4 years ago

I used to disable the entire node size machinery when cross compiling precisely to catch such issues, but that makes it impossible to use it in cross compiling at all. I'm open for PRs that improve the behavior, as I don't know anything about cross compiling with CMake.

razr commented 4 years ago

Hello @foonathan, I have provided a fix, it works on my Ubuntu 18.04 while native and cross-compilation. However, I see this issue later on:

[ 45%] Building CXX object example/CMakeFiles/foonathan_memory_example_tracking.dir/tracking.cpp.obj
/home/akholodn/github/razr/memory/example/tracking.cpp:53:70: error: no member named 'set_node_size' in namespace 'foonathan::memory'
                                       memory::memory_pool<>(memory::set_node_size<int>::value,
                                                             ~~~~~~~~^
/home/akholodn/github/razr/memory/example/tracking.cpp:53:87: error: expected '(' for function-style cast or type construction
                                       memory::memory_pool<>(memory::set_node_size<int>::value,
                                                                                   ~~~^
/home/akholodn/github/razr/memory/example/tracking.cpp:53:90: error: no member named 'value' in the global namespace
                                       memory::memory_pool<>(memory::set_node_size<int>::value,
                                                                                       ~~^
foonathan commented 4 years ago

It seems it didn't generate the node size header from the tool correctly. But as your PR works on CI, it might be a local issue. Have you tried clearing your build folder and running everything again?

razr commented 4 years ago

Native compile on my Ubuntu 18.04 machine fails somewhere here. I'm on your latest changes.

akholodn@razilix:~/github/razr/memory/build$ git log
commit de1a887d2049b7dc92efd38deae9fad230b06528 (HEAD -> master, upstream/master, origin/master, origin/HEAD)
Author: Jonathan Müller <git@foonathan.net>
Date:   Fri May 29 18:34:13 2020 +0200

    Fix tracked_allocator for allocators without block allocators

    Fixes #79.
akholodn@razilix:~/github/razr/memory/build$ cmake ..
-- The C compiler identification is GNU 8.4.0
-- The CXX compiler identification is GNU 8.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/akholodn/github/razr/memory/build
akholodn@razilix:~/github/razr/memory/build$ make
Scanning dependencies of target foonathan_memory_node_size_debugger
[  1%] Building CXX object tool/CMakeFiles/foonathan_memory_node_size_debugger.dir/node_size_debugger.cpp.o
[  3%] Linking CXX executable nodesize_dbg
[  3%] Built target foonathan_memory_node_size_debugger
[  5%] Generating container_node_sizes_impl.hpp
Scanning dependencies of target foonathan_memory
[  7%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/align.cpp.o
[  8%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/debug_helpers.cpp.o
[ 10%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/assert.cpp.o
[ 12%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/free_list.cpp.o
[ 14%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/free_list_array.cpp.o
[ 16%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/small_free_list.cpp.o
[ 17%] Building CXX object src/CMakeFiles/foonathan_memory.dir/debugging.cpp.o
[ 19%] Building CXX object src/CMakeFiles/foonathan_memory.dir/error.cpp.o
[ 21%] Building CXX object src/CMakeFiles/foonathan_memory.dir/heap_allocator.cpp.o
[ 23%] Building CXX object src/CMakeFiles/foonathan_memory.dir/iteration_allocator.cpp.o
[ 25%] Building CXX object src/CMakeFiles/foonathan_memory.dir/malloc_allocator.cpp.o
[ 26%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_arena.cpp.o
[ 28%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_pool.cpp.o
[ 30%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_pool_collection.cpp.o
[ 32%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_stack.cpp.o
[ 33%] Building CXX object src/CMakeFiles/foonathan_memory.dir/new_allocator.cpp.o
[ 35%] Building CXX object src/CMakeFiles/foonathan_memory.dir/static_allocator.cpp.o
[ 37%] Building CXX object src/CMakeFiles/foonathan_memory.dir/temporary_allocator.cpp.o
[ 39%] Building CXX object src/CMakeFiles/foonathan_memory.dir/virtual_memory.cpp.o
[ 41%] Linking CXX static library libfoonathan_memory-0.6.2.a
[ 41%] Built target foonathan_memory
Scanning dependencies of target foonathan_memory_example_joint_allocation
[ 42%] Building CXX object example/CMakeFiles/foonathan_memory_example_joint_allocation.dir/joint_allocation.cpp.o
[ 44%] Linking CXX executable foonathan_memory_example_joint_allocation
[ 44%] Built target foonathan_memory_example_joint_allocation
Scanning dependencies of target foonathan_memory_example_allocator_storage
[ 46%] Building CXX object example/CMakeFiles/foonathan_memory_example_allocator_storage.dir/allocator_storage.cpp.o
[ 48%] Linking CXX executable foonathan_memory_example_allocator_storage
[ 48%] Built target foonathan_memory_example_allocator_storage
Scanning dependencies of target foonathan_memory_example_taking_allocators
[ 50%] Building CXX object example/CMakeFiles/foonathan_memory_example_taking_allocators.dir/taking_allocators.cpp.o
[ 51%] Linking CXX executable foonathan_memory_example_taking_allocators
[ 51%] Built target foonathan_memory_example_taking_allocators
Scanning dependencies of target foonathan_memory_example_tracking
[ 53%] Building CXX object example/CMakeFiles/foonathan_memory_example_tracking.dir/tracking.cpp.o
[ 55%] Linking CXX executable foonathan_memory_example_tracking
[ 55%] Built target foonathan_memory_example_tracking
Scanning dependencies of target foonathan_memory_example_using_allocators
[ 57%] Building CXX object example/CMakeFiles/foonathan_memory_example_using_allocators.dir/using_allocators.cpp.o
[ 58%] Linking CXX executable foonathan_memory_example_using_allocators
[ 58%] Built target foonathan_memory_example_using_allocators
Scanning dependencies of target foonathan_memory_test
[ 60%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/test.cpp.o
[ 62%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/detail/align.cpp.o
[ 64%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/detail/debug_helpers.cpp.o
[ 66%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/detail/free_list.cpp.o
[ 67%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/detail/free_list_array.cpp.o
[ 69%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/detail/ilog2.cpp.o
[ 71%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/detail/memory_stack.cpp.o
[ 73%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/aligned_allocator.cpp.o
[ 75%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/allocator_traits.cpp.o
[ 76%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/default_allocator.cpp.o
[ 78%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/fallback_allocator.cpp.o
[ 80%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/iteration_allocator.cpp.o
[ 82%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/joint_allocator.cpp.o
[ 83%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/memory_arena.cpp.o
[ 85%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/memory_pool.cpp.o
[ 87%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/memory_pool_collection.cpp.o
[ 89%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/memory_resource_adapter.cpp.o
In file included from /home/akholodn/github/razr/memory/include/foonathan/memory/memory_resource_adapter.hpp:23,
                 from /home/akholodn/github/razr/memory/test/memory_resource_adapter.cpp:5:
/usr/include/c++/8/experimental/memory_resource:233:11: error: expected primary-expression before ‘auto’
  decltype(auto)
           ^~~~
/usr/include/c++/8/experimental/memory_resource:233:11: error: expected ‘)’ before ‘auto’
  decltype(auto)
          ~^~~~
           )
/usr/include/c++/8/experimental/memory_resource:233:11: error: expected primary-expression before ‘auto’
  decltype(auto)
           ^~~~
/usr/include/c++/8/experimental/memory_resource:233:11: error: expected primary-expression before ‘auto’
/usr/include/c++/8/experimental/memory_resource:233:11: error: expected primary-expression before ‘auto’
/usr/include/c++/8/experimental/memory_resource:233:11: error: expected primary-expression before ‘auto’
/usr/include/c++/8/experimental/memory_resource:233:2: error: expected unqualified-id before ‘decltype’
  decltype(auto)
  ^~~~~~~~
/usr/include/c++/8/experimental/memory_resource:239:11: error: expected primary-expression before ‘auto’
  decltype(auto)
           ^~~~
/usr/include/c++/8/experimental/memory_resource:239:11: error: expected ‘)’ before ‘auto’
  decltype(auto)
          ~^~~~
           )
/usr/include/c++/8/experimental/memory_resource:239:11: error: expected primary-expression before ‘auto’
  decltype(auto)
           ^~~~
/usr/include/c++/8/experimental/memory_resource:239:11: error: expected primary-expression before ‘auto’
/usr/include/c++/8/experimental/memory_resource:239:11: error: expected primary-expression before ‘auto’
/usr/include/c++/8/experimental/memory_resource:239:11: error: expected primary-expression before ‘auto’
/usr/include/c++/8/experimental/memory_resource:239:2: error: expected unqualified-id before ‘decltype’
  decltype(auto)
  ^~~~~~~~
test/CMakeFiles/foonathan_memory_test.dir/build.make:446: recipe for target 'test/CMakeFiles/foonathan_memory_test.dir/memory_resource_adapter.cpp.o' failed
make[2]: *** [test/CMakeFiles/foonathan_memory_test.dir/memory_resource_adapter.cpp.o] Error 1
CMakeFiles/Makefile2:353: recipe for target 'test/CMakeFiles/foonathan_memory_test.dir/all' failed
make[1]: *** [test/CMakeFiles/foonathan_memory_test.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2
razr commented 4 years ago

And the cross-compilation fails here. This is most probably because of FOONATHAN_MEMORY_NO_NODE_SIZE is defined but not added to the memory/example/tracking.cpp? #ifdef FOONATHAN_MEMORY_NO_NODE_SIZE do_something different?

akholodn@razilix:~/github/razr/memory/vxworks-build$ cat ./src/container_node_sizes_impl.hpp
#define FOONATHAN_MEMORY_NO_NODE_SIZE
akholodn@razilix:~/github/razr/memory/vxworks-build$ cmake  .. -DCMAKE_TOOLCHAIN_FILE=$HOME/stash/users/akholodn/cmake-test/buildspecs/cmake/toolchain.cmake
-- The C compiler identification is Clang 9.0.1
-- The CXX compiler identification is Clang 9.0.1
-- Check for working C compiler: /home/akholodn/Downloads/wrsdk-vxworks7-qemu/toolkit/host_tools/x86_64-linux/bin/wr-cc
-- Check for working C compiler: /home/akholodn/Downloads/wrsdk-vxworks7-qemu/toolkit/host_tools/x86_64-linux/bin/wr-cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/akholodn/Downloads/wrsdk-vxworks7-qemu/toolkit/host_tools/x86_64-linux/bin/wr-c++
-- Check for working CXX compiler: /home/akholodn/Downloads/wrsdk-vxworks7-qemu/toolkit/host_tools/x86_64-linux/bin/wr-c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning at src/CMakeLists.txt:93 (message):
  cross-compiling, but emulator is not defined, cannot generate
  container_node_sizes_impl.hpp, node size information will be unavailable

-- Configuring done
-- Generating done
-- Build files have been written to: /home/akholodn/github/razr/memory/vxworks-build
akholodn@razilix:~/github/razr/memory/vxworks-build$
akholodn@razilix:~/github/razr/memory/vxworks-build$
akholodn@razilix:~/github/razr/memory/vxworks-build$ cat ./src/container_node_sizes_impl.hpp
#define FOONATHAN_MEMORY_NO_NODE_SIZE
akholodn@razilix:~/github/razr/memory/vxworks-build$
akholodn@razilix:~/github/razr/memory/vxworks-build$ make
Scanning dependencies of target foonathan_memory
[  1%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/align.cpp.o
[  3%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/debug_helpers.cpp.o
[  5%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/assert.cpp.o
[  7%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/free_list.cpp.o
[  9%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/free_list_array.cpp.o
[ 10%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/small_free_list.cpp.o
[ 12%] Building CXX object src/CMakeFiles/foonathan_memory.dir/debugging.cpp.o
[ 14%] Building CXX object src/CMakeFiles/foonathan_memory.dir/error.cpp.o
[ 16%] Building CXX object src/CMakeFiles/foonathan_memory.dir/heap_allocator.cpp.o
[ 18%] Building CXX object src/CMakeFiles/foonathan_memory.dir/iteration_allocator.cpp.o
[ 20%] Building CXX object src/CMakeFiles/foonathan_memory.dir/malloc_allocator.cpp.o
[ 21%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_arena.cpp.o
[ 23%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_pool.cpp.o
[ 25%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_pool_collection.cpp.o
[ 27%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_stack.cpp.o
[ 29%] Building CXX object src/CMakeFiles/foonathan_memory.dir/new_allocator.cpp.o
[ 30%] Building CXX object src/CMakeFiles/foonathan_memory.dir/static_allocator.cpp.o
[ 32%] Building CXX object src/CMakeFiles/foonathan_memory.dir/temporary_allocator.cpp.o
[ 34%] Building CXX object src/CMakeFiles/foonathan_memory.dir/virtual_memory.cpp.o
[ 36%] Linking CXX static library libfoonathan_memory-0.6.2.a
[ 36%] Built target foonathan_memory
Scanning dependencies of target foonathan_memory_example_taking_allocators
[ 38%] Building CXX object example/CMakeFiles/foonathan_memory_example_taking_allocators.dir/taking_allocators.cpp.o
[ 40%] Linking CXX executable foonathan_memory_example_taking_allocators
[ 40%] Built target foonathan_memory_example_taking_allocators
Scanning dependencies of target foonathan_memory_example_allocator_storage
[ 41%] Building CXX object example/CMakeFiles/foonathan_memory_example_allocator_storage.dir/allocator_storage.cpp.o
[ 43%] Linking CXX executable foonathan_memory_example_allocator_storage
[ 43%] Built target foonathan_memory_example_allocator_storage
Scanning dependencies of target foonathan_memory_example_tracking
[ 45%] Building CXX object example/CMakeFiles/foonathan_memory_example_tracking.dir/tracking.cpp.o
/home/akholodn/github/razr/memory/example/tracking.cpp:68:74: error: no member named 'set_node_size' in namespace 'foonathan::memory'
                                           memory::memory_pool<>(memory::set_node_size<int>::value,
                                                                 ~~~~~~~~^
/home/akholodn/github/razr/memory/example/tracking.cpp:68:91: error: expected '(' for function-style cast or type construction
                                           memory::memory_pool<>(memory::set_node_size<int>::value,
                                                                                       ~~~^
/home/akholodn/github/razr/memory/example/tracking.cpp:68:94: error: no member named 'value' in the global namespace
                                           memory::memory_pool<>(memory::set_node_size<int>::value,
                                                                                           ~~^
3 errors generated.
example/CMakeFiles/foonathan_memory_example_tracking.dir/build.make:62: recipe for target 'example/CMakeFiles/foonathan_memory_example_tracking.dir/tracking.cpp.o' failed
make[2]: *** [example/CMakeFiles/foonathan_memory_example_tracking.dir/tracking.cpp.o] Error 1
CMakeFiles/Makefile2:222: recipe for target 'example/CMakeFiles/foonathan_memory_example_tracking.dir/all' failed
make[1]: *** [example/CMakeFiles/foonathan_memory_example_tracking.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2
foonathan commented 4 years ago

The first error is unrelated, but annoying: your standard library provides the memory_resource header in C++11 mode, but it doesn't compile when included unless C++17 is activated. I should probably rethink the feature check there.

The second error is as expected. I will disable examples when the node sizes are not available.

razr commented 4 years ago

yes, that's right, I just would like to make you aware that there is a problem with it :). btw, is it not possible to calculate node sizes during a runtime? Then you maybe don't need this cross-compiling logic for the header generation.

foonathan commented 4 years ago

Both error should be fixed now, please confirm.

btw, is it not possible to calculate node sizes during a runtime? Then you maybe don't need this cross-compiling logic for the header generation.

I mean, it is - that's how the tool works. But it requires some sort of heap allocation at startup to figure it out, or something. The tool approach seems easier.

razr commented 4 years ago

a native compilation works, the cross-compilation still fails on my side, but I think it is due to error in the C++ library:

akholodn@windix:~/github/razr/memory/vxworks-build$ make
[ 39%] Built target foonathan_memory
[ 43%] Built target foonathan_memory_example_taking_allocators
[ 47%] Built target foonathan_memory_example_joint_allocation
[ 50%] Built target foonathan_memory_example_allocator_storage
[ 52%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/memory_resource_adapter.cpp.obj
In file included from /home/akholodn/github/razr/memory/test/memory_resource_adapter.cpp:5:
In file included from /home/akholodn/github/razr/memory/include/foonathan/memory/memory_resource_adapter.hpp:18:
/home/akholodn/Downloads/wrsdk-vxworks7-qemu/toolkit/include/usr/h/public/memory_resource:275:11: error: no member named 'try_emplace' in 'std::map<unsigned long, std::pmr::_Chunk_vec, std::less<unsigned long>, std::allocator<std::pair<const unsigned long, std::pmr::_Chunk_vec> > >'
                        _Mymap.try_emplace(_Mymaptype::iterator(), _Block_size,
                        ~~~~~~ ^
1 error generated.
make[2]: *** [test/CMakeFiles/foonathan_memory_test.dir/build.make:447: test/CMakeFiles/foonathan_memory_test.dir/memory_resource_adapter.cpp.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:276: test/CMakeFiles/foonathan_memory_test.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
mdnasr commented 4 years ago

I have tested the changes with Yocto for Beagle Bone Black. Cross compiling successfully worked but failed when being used. I am trying to build FastRTPS which uses this library, it failed with the following errors while cross compiling:

| /build-fastrtps/tmp/work/i586-poky-linux/fastrtps/1.0-r0/git/src/cpp/rtps/reader/WriterProxy.cpp:56:46: error: 'set_node_size' is not a member of 'memory'
|  constexpr size_t changes_node_size = memory::set_node_size<std::pair<size_t, SequenceNumber_t> >::value;
|                                               ^~~~~~~~~~~~~
| /build-fastrtps/tmp/work/i586-poky-linux/fastrtps/1.0-r0/git/src/cpp/rtps/reader/WriterProxy.cpp:56:46: note: suggested alternative: 'bad_node_size'
|  constexpr size_t changes_node_size = memory::set_node_size<std::pair<size_t, SequenceNumber_t> >::value;
|                                               ^~~~~~~~~~~~~
|                                               bad_node_size
| /build-fastrtps/tmp/work/i586-poky-linux/fastrtps/1.0-r0/git/src/cpp/rtps/reader/WriterProxy.cpp:56:96: error: expected primary-expression before '>' token
|  constexpr size_t changes_node_size = memory::set_node_size<std::pair<size_t, SequenceNumber_t> >::value;
|                                                                                                 ^
| /build-fastrtps/tmp/work/i586-poky-linux/fastrtps/1.0-r0/git/src/cpp/rtps/reader/WriterProxy.cpp:56:99: error: '::value' has not been declared
|  constexpr size_t changes_node_size = memory::set_node_size<std::pair<size_t, SequenceNumber_t> >::value;
|                                                                                                    ^~~~~
| src/cpp/CMakeFiles/fastrtps.dir/build.make:665: recipe for target 'src/cpp/CMakeFiles/fastrtps.dir/rtps/reader/WriterProxy.cpp.o' failed
| make[2]: *** [src/cpp/CMakeFiles/fastrtps.dir/rtps/reader/WriterProxy.cpp.o] Error 1
| make[2]: *** Waiting for unfinished jobs....
| In file included from /build-fastrtps/tmp/work/i586-poky-linux/fastrtps/1.0-r0/git/src/cpp/rtps/reader/RTPSReader.cpp:23:0:
| /build-fastrtps/tmp/work/i586-poky-linux/fastrtps/1.0-r0/git/src/cpp/rtps/reader/ReaderHistoryState.hpp:37:24: error: 'map_node_size' is not a member of 'foonathan::memory'
|      foonathan::memory::map_node_size<std::pair<size_t, std::pair<GUID_t, GUID_t>>>::value;
|                         ^~~~~~~~~~~~~
| /build-fastrtps/tmp/work/i586-poky-linux/fastrtps/1.0-r0/git/src/cpp/rtps/reader/ReaderHistoryState.hpp:37:24: note: suggested alternative: 'bad_node_size'
|      foonathan::memory::map_node_size<std::pair<size_t, std::pair<GUID_t, GUID_t>>>::value;
|                         ^~~~~~~~~~~~~
|                         bad_node_size

I have found that /usr/include/foonathan_memory/foonathan/memory/detail/container_node_sizes_impl.hpp only contains the following line:

#define FOONATHAN_MEMORY_NO_NODE_SIZE

I think this header file should contains the missing data

razr commented 4 years ago
mdnasr commented 4 years ago

@razr, Thanks for feedback. No, Yocto does not define CMAKE_CROSSCOMPILING_EMULATOR and I couldn't find a way to do so.

razr commented 4 years ago

@mdnasr, thanks for doing it. I have looked in your build output and it is so that you cross-compile Linux/Intel (i586-poky-linux) on Linux/Intel host, so you were able to run your cross-compiled _nodesizedbg binary natively to produce a _container_node_sizesimpl.hpp. It worked because the host and the target architectures as well as OSes were the same. E.g. I could not do it because my target OS is VxWorks. With the recent change, you need to define a CMAKE_CROSSCOMPILING_EMULATOR e.g. qemu-x86_64-static, like -DCMAKE_CROSSCOMPILING_EMULATOR=qemu-x86_64-static and pass it to your build. What fastrtps Yocto recipes do you use, could you please paste a link here?

mdnasr commented 4 years ago

@razr The build for Raspberry Pi also succeed with me.

Here is the recipe:

SECTION = "devel"
LICENSE = "CLOSED"

DEPENDS += " \
    asio \
    libtinyxml2 \
"

SRC_URI = "git://github.com/foonathan/memory.git"
SRCREV="${AUTOREV}"

S = "${WORKDIR}/git"

inherit cmake pkgconfig

FILES_${PN} += " ${libdir}/* ${bindir}/* ${datadir}/*"
SYSROOT_DIRS += "${bindir}"

and here is output when building this recipe that shows the target information

Loading cache: 100% |#################################################################################################################################| Time: 0:00:00
Loaded 2868 entries from dependency cache.
Parsing recipes: 100% |###############################################################################################################################| Time: 0:00:02
Parsing of 2115 .bb files complete (2062 cached, 53 parsed). 2921 targets, 110 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.36.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "arm-poky-linux-gnueabi"
MACHINE              = "raspberrypi3"
DISTRO               = "poky"
DISTRO_VERSION       = "2.4.4"
TUNE_FEATURES        = "arm armv7ve vfp thumb neon vfpv4 callconvention-hard cortexa7"
TARGET_FPU           = "hard"
meta                 
meta-poky            
meta-yocto-bsp       = "rocko:5f660914cd7eec8117efccdf1eb29c466b4e74f7"
meta-oe              
meta-python          
meta-multimedia      
meta-networking      = "rocko:eae996301d9c097bcbeb8046f08041dc82bb62f8"
meta-raspberrypi     = "rocko:8e4c537d84fdde8e3b4642d0dda2c0f4af76d52f"
meta-ara             
meta-st              = "ave-master:9229801be1a568fce46e1b228219398e618b02f7"
meta-ave             = "ave-master:88d9be695867bc3c135b602d788bae4ae4c86617"

Initialising tasks: 100% |############################################################################################################################| Time: 0:00:01
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 875 tasks of which 860 didn't need to be rerun and all succeeded.

Also I have found bin/nodesize_dbg elf file in output package in ARM format. and still usr/include/foonathan_memory/foonathan/memory/detail/container_node_sizes_impl.hpp only defines this #define FOONATHAN_MEMORY_NO_NODE_SIZE. I am little confused of this finding and don't know what it actually mean!!

 ~/build_rpi/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/foonathan-memory-vendor/1.0-r0/package/usr/bin > file nodesize_dbg
nodesize_dbg: ELF 32-bit LSB executable, ARM, EABI5 version 1 (GNU/Linux), statically linked, for GNU/Linux 3.2.0, BuildID[sha1]=89a0e4359d02da476c4f5f8ba608802b3218eb42, stripped
razr commented 4 years ago

This is the correct behavior. nodesize_dbg is built for ARM, and you can't run it on your Intel host, that's why you get #define FOONATHAN_MEMORY_NO_NODE_SIZE. Now if you would define in your recipe something like EXTRA_OECMAKE = "-DCMAKE_CROSSCOMPILING_EMULATOR=qemu$ARCH" then container_node_sizes_impl.hpp shall be properly generated. qemu$ARCH should show to your qemuarm or qemux86-64 respectively.

mdnasr commented 4 years ago

It fails because qemuarm is not found. I also appended qemu to DEPENDS variable with no success. How did you get qemu to be installed in recipe sysroot?

razr commented 4 years ago

you should build qemu-native, please try to add 

NATIVEDEPS += "qemu-native"

or

DEPENDS += "qemu-native"

and you could probably select a target you need, see https://git.yoctoproject.org/cgit.cgi/poky/plain/meta/recipes-devtools/qemu/qemu.inc 

QEMU_TARGETS="aarch64 x86_64"
foonathan commented 4 years ago

a native compilation works, the cross-compilation still fails on my side, but I think it is due to error in the C++ library:

akholodn@windix:~/github/razr/memory/vxworks-build$ make
[ 39%] Built target foonathan_memory
[ 43%] Built target foonathan_memory_example_taking_allocators
[ 47%] Built target foonathan_memory_example_joint_allocation
[ 50%] Built target foonathan_memory_example_allocator_storage
[ 52%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/memory_resource_adapter.cpp.obj
In file included from /home/akholodn/github/razr/memory/test/memory_resource_adapter.cpp:5:
In file included from /home/akholodn/github/razr/memory/include/foonathan/memory/memory_resource_adapter.hpp:18:
/home/akholodn/Downloads/wrsdk-vxworks7-qemu/toolkit/include/usr/h/public/memory_resource:275:11: error: no member named 'try_emplace' in 'std::map<unsigned long, std::pmr::_Chunk_vec, std::less<unsigned long>, std::allocator<std::pair<const unsigned long, std::pmr::_Chunk_vec> > >'
                        _Mymap.try_emplace(_Mymaptype::iterator(), _Block_size,
                        ~~~~~~ ^
1 error generated.
make[2]: *** [test/CMakeFiles/foonathan_memory_test.dir/build.make:447: test/CMakeFiles/foonathan_memory_test.dir/memory_resource_adapter.cpp.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:276: test/CMakeFiles/foonathan_memory_test.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

Should be fixed on master.

@all: Feel free to discuss cross compiling issues here, but please open new issues if there is anything I need to do. Thanks!

asad26 commented 1 year ago

Hi @MiguelCompany I have a question related to cross compilation emulator for QNX, not exactly related to this thread. I cross compiled a binary containing google tests for QNX 7.1.0 on Ubuntu Focal. When I try to use qemu-aarch64 emulator to run the binary, it results in segmentation fault error.

You mentioned here that the CMAKE_CROSSCOMPILING_EMULATOR variable needs to be set. Can you tell me which emulator to use if I want to run QNX binary on x86_64 system? Any help would be much appreciated. Thanks