search

spinicist / riesling

Radial Interstices Enable Speedy Low-Volume Imaging
MIT License
24 stars 9 forks source link

Issue with stray characters when compiling #41

Open MartinK84 opened 1 year ago

MartinK84 commented 1 year ago

I was trying the latest version and unfortunately running into a problem when compilation (can't use the pre compiled binaries as our Linux systems are too ancient). The problem is that you are apparently using utf-8 variables in the code and I'm thus getting tons of errors like stray »\316« in program. Do you have any advice in how to fix this? Are there any compile flags that could be used/tweaked?

fyrdahl commented 1 year ago

I'm going through similar issues,

What I found so far is that the minimum required CMake-version is 3.13, contrary to CMakeLists.txt https://github.com/spinicist/riesling/blob/96b3e81d62658a59d9a8c321cf22dabb317ee751/CMakeLists.txt#L1

The other thing I found is that the compiler has to be gcc-10 or higher, but no luck yet.

Edit: I was still using GCC 9.4 from the system. When specifying the correct gcc version I got it to build. I added -DCMAKE_C_COMPILER=gcc-10 -DCMAKE_CXX_COMPILER=g++-10 in bootstrap.sh.

MartinK84 commented 1 year ago

Thank you very much, upgrading cmake and going with gcc-10 fixed the problem and I was able to compile it on our only Ubuntu 20.04 system (did not work on our 18.04 or CentOS Stream 8 servers). Running the compiled binary on the 20.04 system it still died with Illegal Instruction (core dumped) but after adding

set(CMAKE_FIND_LIBRARY_SUFFIXES ".a")
set(BUILD_SHARED_LIBS OFF)
set(CMAKE_EXE_LINKER_FLAGS "-static")

to the CMakeLists.txt it did compile static enough so that the binary compiled at 20.04 and would then also run at our computation servers running 18.04.

Maybe since compilation seems to be very tricky, it would be great to include instructions on how to compile static and maybe also build the release binaries here on GitHub as a static build (the release binaries did not run on any of our 18.04, 20.04 or CentOS 8 Stream systems)

spinicist commented 1 year ago

Hello - thanks to both of you for the above.

As you may have noticed, I have not had time to update the documentation, which I do regret massively. I have a lot to do. You are correct that GCC10 is now required. I didn't realise that something in CMakeLists needs 3.13 now, I wonder what that is.

The Github 18.04 runner is now deprecated. I made the mistake of jumping all the way to 22.04 as you can see here: https://github.com/spinicist/riesling/blob/main/.github/workflows/build.yml. I think I should downgrade that to 20.04 and re-tag the v0.9 release, that ought to mean you can run them on 20.04 (and possibly 18.04 given what @MartinK84 said above).

spinicist commented 1 year ago

What I found so far is that the minimum required CMake-version is 3.13, contrary to CMakeLists.txt

I will happily accept a Pull Request updating this.

spinicist commented 1 year ago

to the CMakeLists.txt it did compile static enough so that the binary compiled at 20.04 and would then also run at our computation servers running 18.04.

This I am confused by, because my vcpkg triplet should be forcing static linking as specified here: https://github.com/spinicist/cmake/blob/108091e5cecfa6a0a38b0fbfbf41f0fbf487618f/triplets/x64-linux.cmake#L6

Do you use ninja as the build system or Make? If it is ninja and you have time, it might be instructive to try recompiling without your changes using ninja -v and then posting the final link command here.

MartinK84 commented 1 year ago

to the CMakeLists.txt it did compile static enough so that the binary compiled at 20.04 and would then also run at our computation servers running 18.04.

This I am confused by, because my vcpkg triplet should be forcing static linking as specified here: https://github.com/spinicist/cmake/blob/108091e5cecfa6a0a38b0fbfbf41f0fbf487618f/triplets/x64-linux.cmake#L6

Do you use ninja as the build system or Make? If it is ninja and you have time, it might be instructive to try recompiling without your changes using ninja -v and then posting the final link command here.

Without the changes the output is:

[0/1] /usr/bin/cmake -S.../riesling -B.../riesling/build
-- Running vcpkg install
-- Running vcpkg install - done
-- Git Version: v0.9-dirty Timestamp: 2023-01-18T13:15:30Z
-- Configuring done
-- Generating done
-- Build files have been written to: .../riesling/build
[1/3] : && /usr/bin/g++-10  -Wall -Wpedantic -Wshadow -O3 -DNDEBUG  -Wl,--as-needed CMakeFiles/riesling-tests.dir/test/cropper.cpp.o CMakeFiles/riesling-tests.dir/test/decomp.cpp.o CMakeFiles/riesling-tests.dir/test/fft3.cpp.o CMakeFiles/riesling-tests.dir/test/io.cpp.o CMakeFiles/riesling-tests.dir/test/kernel.cpp.o CMakeFiles/riesling-tests.dir/test/parameters.cpp.o CMakeFiles/riesling-tests.dir/test/precond.cpp.o CMakeFiles/riesling-tests.dir/test/sdc.cpp.o CMakeFiles/riesling-tests.dir/test/zinfandel.cpp.o CMakeFiles/riesling-tests.dir/test/op/fft.cpp.o CMakeFiles/riesling-tests.dir/test/op/grid.cpp.o CMakeFiles/riesling-tests.dir/test/op/nufft.cpp.o CMakeFiles/riesling-tests.dir/test/op/pad.cpp.o CMakeFiles/riesling-tests.dir/test/op/recon.cpp.o CMakeFiles/riesling-tests.dir/test/op/sense.cpp.o  -o riesling-tests  libvineyard.a  vcpkg_installed/x64-linux/lib/manual-link/libCatch2Main.a  vcpkg_installed/x64-linux/lib/libfftw3f.a  vcpkg_installed/x64-linux/lib/libhdf5.a  -lm  -ldl  -lpthread  vcpkg_installed/x64-linux/lib/libniftiio.a  vcpkg_installed/x64-linux/lib/libznz.a  vcpkg_installed/x64-linux/lib/libz.a  -lm  vcpkg_installed/x64-linux/lib/libscn.a  vcpkg_installed/x64-linux/lib/libCatch2.a && :
[2/3] /usr/bin/g++-10  -DEIGEN_USE_THREADS -DFMT_HEADER_ONLY=1 -DSCN_HEADER_ONLY=0 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -D_POSIX_C_SOURCE=200809L -I. -I../src -isystem vcpkg_installed/x64-linux/share/eigen3/../../include/eigen3 -isystem vcpkg_installed/x64-linux/include -isystem vcpkg_installed/x64-linux/include/nifti -Wall -Wpedantic -Wshadow -O3 -DNDEBUG -fvisibility=hidden   -std=c++2a -MD -MT CMakeFiles/riesling.dir/src/cmd/version.cpp.o -MF CMakeFiles/riesling.dir/src/cmd/version.cpp.o.d -o CMakeFiles/riesling.dir/src/cmd/version.cpp.o -c ../src/cmd/version.cpp
[3/3] : && /usr/bin/g++-10  -Wall -Wpedantic -Wshadow -O3 -DNDEBUG  -Wl,--as-needed CMakeFiles/riesling.dir/src/cmd/admm.cpp.o CMakeFiles/riesling.dir/src/cmd/blend.cpp.o CMakeFiles/riesling.dir/src/cmd/cg.cpp.o CMakeFiles/riesling.dir/src/cmd/compress.cpp.o CMakeFiles/riesling.dir/src/cmd/downsamp.cpp.o CMakeFiles/riesling.dir/src/cmd/eig.cpp.o CMakeFiles/riesling.dir/src/cmd/espirit.cpp.o CMakeFiles/riesling.dir/src/cmd/filter.cpp.o CMakeFiles/riesling.dir/src/cmd/frames.cpp.o CMakeFiles/riesling.dir/src/cmd/grid.cpp.o CMakeFiles/riesling.dir/src/cmd/h5.cpp.o CMakeFiles/riesling.dir/src/cmd/lookup.cpp.o CMakeFiles/riesling.dir/src/cmd/lsmr.cpp.o CMakeFiles/riesling.dir/src/cmd/lsqr.cpp.o CMakeFiles/riesling.dir/src/cmd/meta.cpp.o CMakeFiles/riesling.dir/src/cmd/nii.cpp.o CMakeFiles/riesling.dir/src/cmd/noisify.cpp.o CMakeFiles/riesling.dir/src/cmd/nufft.cpp.o CMakeFiles/riesling.dir/src/cmd/pad.cpp.o CMakeFiles/riesling.dir/src/cmd/pdhg.cpp.o CMakeFiles/riesling.dir/src/cmd/phantom.cpp.o CMakeFiles/riesling.dir/src/cmd/plan.cpp.o CMakeFiles/riesling.dir/src/cmd/precond.cpp.o CMakeFiles/riesling.dir/src/cmd/recon.cpp.o CMakeFiles/riesling.dir/src/cmd/reg.cpp.o CMakeFiles/riesling.dir/src/cmd/rss.cpp.o CMakeFiles/riesling.dir/src/cmd/sdc.cpp.o CMakeFiles/riesling.dir/src/cmd/sense.cpp.o CMakeFiles/riesling.dir/src/cmd/sense-calib.cpp.o CMakeFiles/riesling.dir/src/cmd/sense-sim.cpp.o CMakeFiles/riesling.dir/src/cmd/sim.cpp.o CMakeFiles/riesling.dir/src/cmd/split.cpp.o CMakeFiles/riesling.dir/src/cmd/tgv.cpp.o CMakeFiles/riesling.dir/src/cmd/traj.cpp.o CMakeFiles/riesling.dir/src/cmd/transform.cpp.o CMakeFiles/riesling.dir/src/cmd/version.cpp.o CMakeFiles/riesling.dir/src/main.cpp.o  -o riesling  libvineyard.a  vcpkg_installed/x64-linux/lib/libfftw3f.a  vcpkg_installed/x64-linux/lib/libhdf5.a  -lm  -ldl  -lpthread  vcpkg_installed/x64-linux/lib/libniftiio.a  vcpkg_installed/x64-linux/lib/libznz.a  vcpkg_installed/x64-linux/lib/libz.a  -lm  vcpkg_installed/x64-linux/lib/libscn.a && :

and it will die out when run on another system with different Ubuntu version with:

./riesling: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./riesling)
./riesling: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by ./riesling)
./riesling: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by ./riesling)

with the above added CMakeList.txt changes the ninja log is as follows:

[0/1] /usr/bin/cmake -S.../riesling -B.../riesling/build
-- Running vcpkg install
-- Running vcpkg install - done
-- Git Version: v0.9-dirty Timestamp: 2023-01-18T13:19:31Z
-- Configuring done
-- Generating done
-- Build files have been written to: .../riesling/build
[1/3] : && /usr/bin/g++-10  -Wall -Wpedantic -Wshadow -O3 -DNDEBUG  -static    -Wl,--as-needed CMakeFiles/riesling-tests.dir/test/cropper.cpp.o CMakeFiles/riesling-tests.dir/test/decomp.cpp.o CMakeFiles/riesling-tests.dir/test/fft3.cpp.o CMakeFiles/riesling-tests.dir/test/io.cpp.o CMakeFiles/riesling-tests.dir/test/kernel.cpp.o CMakeFiles/riesling-tests.dir/test/parameters.cpp.o CMakeFiles/riesling-tests.dir/test/precond.cpp.o CMakeFiles/riesling-tests.dir/test/sdc.cpp.o CMakeFiles/riesling-tests.dir/test/zinfandel.cpp.o CMakeFiles/riesling-tests.dir/test/op/fft.cpp.o CMakeFiles/riesling-tests.dir/test/op/grid.cpp.o CMakeFiles/riesling-tests.dir/test/op/nufft.cpp.o CMakeFiles/riesling-tests.dir/test/op/pad.cpp.o CMakeFiles/riesling-tests.dir/test/op/recon.cpp.o CMakeFiles/riesling-tests.dir/test/op/sense.cpp.o  -o riesling-tests  libvineyard.a  vcpkg_installed/x64-linux/lib/manual-link/libCatch2Main.a  vcpkg_installed/x64-linux/lib/libfftw3f.a  vcpkg_installed/x64-linux/lib/libhdf5.a  -lm  -ldl  -lpthread  vcpkg_installed/x64-linux/lib/libniftiio.a  vcpkg_installed/x64-linux/lib/libznz.a  vcpkg_installed/x64-linux/lib/libz.a  -lm  vcpkg_installed/x64-linux/lib/libscn.a  vcpkg_installed/x64-linux/lib/libCatch2.a && :
/usr/bin/ld: vcpkg_installed/x64-linux/lib/libhdf5.a(H5PLint.c.o): in function `H5PL__open':
H5PLint.c:(.text+0x5be): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: libvineyard.a(fft.cpp.o): in function `rl::FFT::WisdomPath[abi:cxx11]()':
fft.cpp:(.text+0x3cd7): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
[2/3] /usr/bin/g++-10  -DEIGEN_USE_THREADS -DFMT_HEADER_ONLY=1 -DSCN_HEADER_ONLY=0 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -D_POSIX_C_SOURCE=200809L -I. -I../src -isystem vcpkg_installed/x64-linux/share/eigen3/../../include/eigen3 -isystem vcpkg_installed/x64-linux/include -isystem vcpkg_installed/x64-linux/include/nifti -Wall -Wpedantic -Wshadow -O3 -DNDEBUG -fvisibility=hidden   -std=c++2a -MD -MT CMakeFiles/riesling.dir/src/cmd/version.cpp.o -MF CMakeFiles/riesling.dir/src/cmd/version.cpp.o.d -o CMakeFiles/riesling.dir/src/cmd/version.cpp.o -c ../src/cmd/version.cpp
[3/3] : && /usr/bin/g++-10  -Wall -Wpedantic -Wshadow -O3 -DNDEBUG  -static    -Wl,--as-needed CMakeFiles/riesling.dir/src/cmd/admm.cpp.o CMakeFiles/riesling.dir/src/cmd/blend.cpp.o CMakeFiles/riesling.dir/src/cmd/cg.cpp.o CMakeFiles/riesling.dir/src/cmd/compress.cpp.o CMakeFiles/riesling.dir/src/cmd/downsamp.cpp.o CMakeFiles/riesling.dir/src/cmd/eig.cpp.o CMakeFiles/riesling.dir/src/cmd/espirit.cpp.o CMakeFiles/riesling.dir/src/cmd/filter.cpp.o CMakeFiles/riesling.dir/src/cmd/frames.cpp.o CMakeFiles/riesling.dir/src/cmd/grid.cpp.o CMakeFiles/riesling.dir/src/cmd/h5.cpp.o CMakeFiles/riesling.dir/src/cmd/lookup.cpp.o CMakeFiles/riesling.dir/src/cmd/lsmr.cpp.o CMakeFiles/riesling.dir/src/cmd/lsqr.cpp.o CMakeFiles/riesling.dir/src/cmd/meta.cpp.o CMakeFiles/riesling.dir/src/cmd/nii.cpp.o CMakeFiles/riesling.dir/src/cmd/noisify.cpp.o CMakeFiles/riesling.dir/src/cmd/nufft.cpp.o CMakeFiles/riesling.dir/src/cmd/pad.cpp.o CMakeFiles/riesling.dir/src/cmd/pdhg.cpp.o CMakeFiles/riesling.dir/src/cmd/phantom.cpp.o CMakeFiles/riesling.dir/src/cmd/plan.cpp.o CMakeFiles/riesling.dir/src/cmd/precond.cpp.o CMakeFiles/riesling.dir/src/cmd/recon.cpp.o CMakeFiles/riesling.dir/src/cmd/reg.cpp.o CMakeFiles/riesling.dir/src/cmd/rss.cpp.o CMakeFiles/riesling.dir/src/cmd/sdc.cpp.o CMakeFiles/riesling.dir/src/cmd/sense.cpp.o CMakeFiles/riesling.dir/src/cmd/sense-calib.cpp.o CMakeFiles/riesling.dir/src/cmd/sense-sim.cpp.o CMakeFiles/riesling.dir/src/cmd/sim.cpp.o CMakeFiles/riesling.dir/src/cmd/split.cpp.o CMakeFiles/riesling.dir/src/cmd/tgv.cpp.o CMakeFiles/riesling.dir/src/cmd/traj.cpp.o CMakeFiles/riesling.dir/src/cmd/transform.cpp.o CMakeFiles/riesling.dir/src/cmd/version.cpp.o CMakeFiles/riesling.dir/src/main.cpp.o  -o riesling  libvineyard.a  vcpkg_installed/x64-linux/lib/libfftw3f.a  vcpkg_installed/x64-linux/lib/libhdf5.a  -lm  -ldl  -lpthread  vcpkg_installed/x64-linux/lib/libniftiio.a  vcpkg_installed/x64-linux/lib/libznz.a  vcpkg_installed/x64-linux/lib/libz.a  -lm  vcpkg_installed/x64-linux/lib/libscn.a && :
/usr/bin/ld: vcpkg_installed/x64-linux/lib/libhdf5.a(H5PLint.c.o): in function `H5PL__open':
H5PLint.c:(.text+0x5be): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: libvineyard.a(fft.cpp.o): in function `rl::FFT::WisdomPath[abi:cxx11]()':
fft.cpp:(.text+0x3cd7): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

and the binary also works at 18.04

MartinK84 commented 1 year ago

On a side note and probably related to my compilation problems it appears as if the riesling which I did compile static on 20.04 and which I'm trying to run on the 18.04 compuation server appears to seg fault out on basically every command...

riesling phantom tut --matrix=64 --snr=50 --channels=4 --verbosity=3
[15:01:34] Welcome to RIESLING
[15:01:34] Command: phantom
[15:01:34] Creating default thread pool with 48 threads
[15:01:34] Using 48 threads
[15:01:34] Using 4096 hi-res spokes
[15:01:34] Matrix Size: [64, 64, 64] Voxel Size: 2 2 2
[15:01:34] Samples: 64 Traces: 4096
Segmentation fault (core dumped)

probably related to that warning above with H5PLint.c and FFT.cpp, looks like it's not fully static :(

spinicist commented 1 year ago

I have retagged and re-released v0.9 using Ubuntu 20.02 and GCC 10. Can you try the resulting binaries on your machines please?

It very much looks like the static C++ runtime linking did not work when you built, but I can't immediately tell why.

fyrdahl commented 1 year ago

What I found so far is that the minimum required CMake-version is 3.13, contrary to CMakeLists.txt

I will happily accept a Pull Request updating this.

Done

fyrdahl commented 1 year ago

I have retagged and re-released v0.9 using Ubuntu 20.02 and GCC 10. Can you try the resulting binaries on your machines please?

Can confirm the new release works on my machine (Ubuntu 20.04)

MartinK84 commented 1 year ago

I have retagged and re-released v0.9 using Ubuntu 20.02 and GCC 10. Can you try the resulting binaries on your machines please?

It very much looks like the static C++ runtime linking did not work when you built, but I can't immediately tell why.

Unfortunately, it does not work on Ubuntu 18.04 because of missing GLIBC requirements, so it definitely looks like to not be completely static.

On our only Ubuntu 20.04 the precompiled also do fail because of Illegal instruction (core dumped), it's just a desktop computer with an old Intel(R) Core(TM) i3-3220 CPU so probably some instruction set is missing, that's why I initially tried to recompile.

spinicist commented 1 year ago

My understanding is that glibc is the one thing you absolutely cannot link statically - hence why building on the earliest version of Linux, which will have the lowest version of glibc, is advisable. glibc is guaranteed to be forward compatible but not backward compatible. One workaround to this is to swap to musl (https://musl.libc.org) instead, but I would need to read up on how to do that.

However, it looked to me like your build isn't statically linking the C++ runtime either, which is the first bit to figure out. We need to make sure -static-libgcc -static-libstdc++ is added to your linker flags. I thought this should come via the triplet, but looks like I am mistaken. Can you add that manually and try again please? Thanks for your patience.

MartinK84 commented 1 year ago

This is kind of strange... if I add target_link_libraries(vineyard PUBLIC -static-libgcc -static-libstdc++ to CMakeLists.txt when compiling on 20.04 it gives the GLIBC errors when running on18.04.

When I add (which supposedly makes everything static) target_link_libraries(vineyard PUBLIC -static-libgcc -static-libstdc++ -static the problem from before happens, it initially runs but then when accessing libhdf5 it crashes.

I think fixing this will be a significant task and probably not worth the trouble for a 4-year-old Ubuntu version. I will now try to somehow bump cmake and install gcc-10 and g++-10 on my Ubuntu 18.04 and see if I can compile it then.

MartinK84 commented 1 year ago

Finally got it to compile under 18.04 and was also able to run the phantom command on one of our computation servers under 18.04 by doing the following:

@spinicist I hope that my particular problem is solved with this, not sure if you want to continue diving into all those build problems for older ubuntu versions. If you want you can close the issue