search

MRtrix3 / mrtrix3

MRtrix3 provides a set of tools to perform various advanced diffusion MRI analyses, including constrained spherical deconvolution (CSD), probabilistic tractography, track-density imaging, and apparent fibre density
http://www.mrtrix.org
Mozilla Public License 2.0
294 stars 181 forks source link

Fix compilation of big object files in MINGW debug builds #2897

Closed daljit46 closed 5 months ago

daljit46 commented 6 months ago

Addresses #2896

github-actions[bot] commented 6 months ago

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] commented 6 months ago

clang-tidy review says "All clean, LGTM! :+1:"

Lestropie commented 6 months ago

Overcame the original issue, but did not compile successfully. Get a massive compiler output; here's some random snippets:

C:/msys64/mingw64/bin/ld: core/CMakeFiles/mrtrix-core.dir/math/stats/glm.cpp.obj:glm.cpp:(.text$_ZN5Eigen15PlainObjectBaseINS_6MatrixIdLin1ELin1ELi1ELin1ELin1EEEE12_set_noaliasINS_7ProductINS_9TransposeINS1_IdLin1ELin1ELi0ELin1ELin1EEEEENS_15DiagonalWrapperIKNS1_IdLin1ELi1ELi0ELin1ELi1EEEEELi1EEEEERS2_RKNS_9DenseBaseIT_EE+0x4b): undefined reference to `void Eigen::internal::call_assignment_no_alias<Eigen::Matrix<double, -1, -1, 1, -1, -1>, Eigen::Product<Eigen::Transpose<Eigen::Matrix<double, -1, -1, 0, -1, -1> >, Eigen::DiagonalWrapper<Eigen::Matrix<double, -1, 1, 0, -1, 1> const>, 1>, Eigen::internal::assign_op<double, double> >(Eigen::Matrix<double, -1, -1, 1, -1, -1>&, Eigen::Product<Eigen::Transpose<Eigen::Matrix<double, -1, -1, 0, -1, -1> >, Eigen::DiagonalWrapper<Eigen::Matrix<double, -1, 1, 0, -1, 1> const>, 1> const&, Eigen::internal::assign_op<double, double> const&)'
...
C:/msys64/mingw64/bin/ld: core/CMakeFiles/mrtrix-core.dir/math/stats/glm.cpp.obj:glm.cpp:(.text$_ZNK5Eigen9DenseBaseINS_13CwiseBinaryOpINS_8internal22scalar_conj_product_opIddEEKNS_9TransposeIKNS_5BlockIKNS6_IKNS_7ProductINS_6MatrixIdLin1ELin1ELi0ELin1ELin1EEENS_7InverseINS7_INS7_INS5_IS9_EENS_15DiagonalWrapperIKNS8_IdLin1ELi1ELi0ELin1ELi1EEEEELi1EEES9_Li0EEEEELi0EEELi1ELin1ELb0EEELi1ELin1ELb1EEEEEKNS6_IKNS5_IKS9_EELin1ELi1ELb0EEEEEE5reduxINS2_13scalar_sum_opIddEEEEdRKT_+0x39): undefined reference to `Eigen::EigenBase<Eigen::CwiseBinaryOp<Eigen::internal::scalar_conj_product_op<double, double>, Eigen::Transpose<Eigen::Block<Eigen::Block<Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Inverse<Eigen::Product<Eigen::Product<Eigen::Transpose<Eigen::Matrix<double, -1, -1, 0, -1, -1> >, Eigen::DiagonalWrapper<Eigen::Matrix<double, -1, 1, 0, -1, 1> const>, 1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> >, 0> const, 1, -1, false> const, 1, -1, true> const> const, Eigen::Block<Eigen::Transpose<Eigen::Matrix<double, -1, -1, 0, -1, -1> const> const, -1, 1, false> const> >::cols() const'
...
C:/msys64/mingw64/bin/ld: core/CMakeFiles/mrtrix-core.dir/math/stats/glm.cpp.obj:glm.cpp:(.text$_ZN5Eigen8internal15unary_evaluatorINS_9TransposeIKNS_5BlockIKNS_7ProductINS4_INS2_INS_13MatrixWrapperINS_5ArrayIdLin1ELi1ELi0ELin1ELi1EEEEEEENS2_IKNS_6MatrixIdLin1ELin1ELi0ELin1ELin1EEEEELi0EEENS_7InverseINS4_INS4_ISB_NSF_INS4_INS4_INS2_ISB_EENS_15DiagonalWrapperIKNSA_IdLin1ELi1ELi0ELin1ELi1EEEEELi1EEESB_Li0EEEEELi0EEESD_Li0EEEEELi0EEELi1ELin1ELb1EEEEENS0_10IndexBasedEdED2Ev+0x14): undefined reference to `Eigen::internal::evaluator<Eigen::Block<Eigen::Product<Eigen::Product<Eigen::Transpose<Eigen::MatrixWrapper<Eigen::Array<double, -1, 1, 0, -1, 1> > >, Eigen::Transpose<Eigen::Matrix<double, -1, -1, 0, -1, -1> const>, 0>, Eigen::Inverse<Eigen::Product<Eigen::Product<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Inverse<Eigen::Product<Eigen::Product<Eigen::Transpose<Eigen::Matrix<double, -1, -1, 0, -1, -1> >, Eigen::DiagonalWrapper<Eigen::Matrix<double, -1, 1, 0, -1, 1> const>, 1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> >, 0>, Eigen::Transpose<Eigen::Matrix<double, -1, -1, 0, -1, -1> const>, 0> >, 0> const, 1, -1, true> const>::~evaluator()'
...
C:/msys64/mingw64/bin/ld: core/CMakeFiles/mrtrix-core.dir/math/stats/glm.cpp.obj:glm.cpp:(.text$_ZSt6fill_nIPjxjET_S1_T0_RKT1_+0x44): undefined reference to `unsigned int* std::__fill_n_a<unsigned int*, long long, unsigned int>(unsigned int*, long long, unsigned int const&, std::random_access_iterator_tag)'
...
C:/msys64/mingw64/bin/ld: core/CMakeFiles/mrtrix-core.dir/math/stats/glm.cpp.obj:glm.cpp:(.rdata$_ZTIN5Eigen6MatrixIdLin1ELin1ELi0ELin1ELin1EEE+0x10): undefined reference to `typeinfo for Eigen::PlainObjectBase<Eigen::Matrix<double, -1, -1, 0, -1, -1> >'
core/CMakeFiles/mrtrix-core.dir/math/stats/glm.cpp.obj:glm.cpp:(.pdata$_ZNK5Eigen7MapBaseINS_5BlockINS_5ArrayIdLin1ELin1ELi0ELin1ELin1EEELin1ELi1ELb1EEELi0EE4rowsEv+0x8): relocation truncated to fit: IMAGE_REL_AMD64_ADDR32NB against `.xdata$_ZNK5Eigen7MapBaseINS_5BlockINS_5ArrayIdLin1ELin1ELi0ELin1ELin1EEELin1ELi1ELb1EEELi0EE4rowsEv'
...
core/CMakeFiles/mrtrix-core.dir/math/stats/glm.cpp.obj:glm.cpp:(.pdata$_ZNK5Eigen15DenseCoeffsBaseINS_12CwiseUnaryOpINS_8internal17scalar_inverse_opIdEEKNS_5BlockINS_5ArrayIdLin1ELin1ELi0ELin1ELin1EEELin1ELi1ELb1EEEEELi0EE5coeffEx+0x4): relocation truncated to fit: IMAGE_REL_AMD64_ADDR32NB against `.text$_ZNK5Eigen15DenseCoeffsBaseINS_12CwiseUnaryOpINS_8internal17scalar_inverse_opIdEEKNS_5BlockINS_5ArrayIdLin1ELin1ELi0ELin1ELin1EEELin1ELi1ELb1EEEEELi0EE5coeffEx'
core/CMakeFiles/mrtrix-core.dir/math/stats/glm.cpp.obj:glm.cpp:(.pdata$_ZNK5Eigen15DenseCoeffsBaseINS_12CwiseUnaryOpINS_8internal17scalar_inverse_opIdEEKNS_5BlockINS_5ArrayIdLin1ELin1ELi0ELin1ELin1EEELin1ELi1ELb1EEEEELi0EE5coeffEx+0x8): additional relocation overflows omitted from the output

Re-running multiple times, it's consistently math/stats/glm.cpp that fails.

While I think it might be a different error message to what I've seen previously, wondering if it could be a 2GB / 4GB RAM filesystem limit.

daljit46 commented 6 months ago

Very strange, I just tried with a fresh installation of MSYS2 and I didn't encounter this problem. This could potentially be because of using the 32-bit version of some package (possibly binutils). I can see that on master we're also using D_FILE_OFFSET_BITS=64, so perhaps you try that (-DCMAKE_CXX_FLAGS="D_FILE_OFFSET_BITS=64")?

Lestropie commented 6 months ago

Good thought, but no success, either on clang or gcc.

Double-checked, and compilation in debug mode still works on this system on master.

Relevant jobs on master vs. here (newlines mine to better see differences):

(1/1) [LD] bin/mrtrix.dll
clang++ \
\
tmp/core/formats/list.o tmp/core/image_io/variable_scaling.o tmp/core/file/dicom/select_cmdline.o tmp/core/formats/nifti1.o tmp/core/stats.o tmp/core/file/dicom/series.o tmp/core/formats/nifti2_gz.o tmp/core/algo/histogram.o tmp/core/math/stats/fwe.o tmp/core/image_io/gz.o tmp/core/misc/bitset.o tmp/core/header.o tmp/core/file/dicom/study.o tmp/core/image_io/scratch.o tmp/core/formats/ram.o tmp/core/file/dicom/image.o tmp/core/file/mmap.o tmp/core/math/zstatistic.o tmp/core/formats/xds.o tmp/core/axes.o tmp/core/formats/par.o tmp/core/image_io/ram.o tmp/core/formats/tiff.o tmp/core/signal_handler.o tmp/core/dwi/shells.o tmp/core/formats/nifti2.o tmp/core/version.o tmp/core/filter/connected_components.o tmp/core/file/key_value.o tmp/core/adapter/reslice.o tmp/core/formats/mrtrix_utils.o tmp/core/file/tiff.o tmp/core/image_io/pipe.o tmp/core/formats/dicom.o tmp/core/progressbar.o tmp/core/datatype.o tmp/core/math/SH.o tmp/core/formats/mrtrix_sparse_legacy.o tmp/core/image_io/sparse.o tmp/core/formats/png.o tmp/core/exception.o tmp/core/formats/mrtrix.o tmp/core/file/json_utils.o tmp/core/image_io/default.o tmp/core/math/stats/glm.o tmp/core/formats/nifti1_gz.o tmp/core/formats/mgz.o tmp/core/math/stats/import.o tmp/core/file/ofstream.o tmp/core/file/nifti_utils.o tmp/core/formats/mri.o tmp/core/image_io/mosaic.o tmp/core/dwi/gradient.o tmp/core/math/bessel.o tmp/core/file/mgh.o tmp/core/formats/mrtrix_gz.o tmp/core/image_io/fetch_store.o tmp/core/file/dicom/dict.o tmp/core/math/betainc.o tmp/core/math/average_space.o tmp/core/file/png.o tmp/core/image_io/tiff.o tmp/core/image_io/base.o tmp/core/thread.o tmp/core/file/dicom/patient.o tmp/core/formats/pipe.o tmp/core/math/stats/shuffle.o tmp/core/file/dicom/mapper.o tmp/core/image_io/png.o tmp/core/file/config.o tmp/core/math/erfinv.o tmp/core/file/name_parser.o tmp/core/formats/mgh.o tmp/core/mrtrix.o tmp/core/phase_encoding.o tmp/core/app.o tmp/core/stride.o tmp/core/file/dicom/quick_scan.o tmp/core/file/dicom/tree.o tmp/core/file/dicom/element.o \
-shared -Wl,--sort-common,--as-needed -pthread -Wl,--allow-multiple-definition -lz -ltiff -lpng16 -lfftw3 -g \
-o bin/mrtrix.dll

-----

[5/221] C:\WINDOWS\system32\cmd.exe /C "cd . && C:\msys64\mingw64\bin\clang++.exe \
-D_FILE_OFFSET_BITS=64 \
-g   -shared \
-o bin\libmrtrix-core.dll \
-Wl,--out-implib,core\libmrtrix-core.dll.a \
-Wl,--major-image-version,0,--minor-image-version,0 \
core/CMakeFiles/mrtrix-core.dir/adapter/reslice.cpp.obj core/CMakeFiles/mrtrix-core.dir/algo/histogram.cpp.obj core/CMakeFiles/mrtrix-core.dir/app.cpp.obj core/CMakeFiles/mrtrix-core.dir/axes.cpp.obj core/CMakeFiles/mrtrix-core.dir/datatype.cpp.obj core/CMakeFiles/mrtrix-core.dir/dwi/gradient.cpp.obj core/CMakeFiles/mrtrix-core.dir/dwi/shells.cpp.obj core/CMakeFiles/mrtrix-core.dir/exception.cpp.obj core/CMakeFiles/mrtrix-core.dir/fetch_store.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/config.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/dicom/dict.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/dicom/element.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/dicom/image.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/dicom/mapper.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/dicom/patient.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/dicom/quick_scan.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/dicom/select_cmdline.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/dicom/series.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/dicom/study.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/dicom/tree.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/json_utils.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/key_value.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/mgh.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/mmap.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/name_parser.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/nifti_utils.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/npy.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/ofstream.cpp.obj core/CMakeFiles/mrtrix-core.dir/file/png.cpp.obj core/CMakeFiles/mrtrix-core.dir/filter/connected_components.cpp.obj core/CMakeFiles/mrtrix-core.dir/fixel/fixel.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/dicom.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/list.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/mgh.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/mgz.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/mri.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/mrtrix.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/mrtrix_gz.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/mrtrix_sparse_legacy.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/mrtrix_utils.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/nifti1.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/nifti1_gz.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/nifti2.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/nifti2_gz.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/par.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/pipe.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/png.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/ram.cpp.obj core/CMakeFiles/mrtrix-core.dir/formats/xds.cpp.obj core/CMakeFiles/mrtrix-core.dir/header.cpp.obj core/CMakeFiles/mrtrix-core.dir/image_io/base.cpp.obj core/CMakeFiles/mrtrix-core.dir/image_io/default.cpp.obj core/CMakeFiles/mrtrix-core.dir/image_io/gz.cpp.obj core/CMakeFiles/mrtrix-core.dir/image_io/mosaic.cpp.obj core/CMakeFiles/mrtrix-core.dir/image_io/pipe.cpp.obj core/CMakeFiles/mrtrix-core.dir/image_io/png.cpp.obj core/CMakeFiles/mrtrix-core.dir/image_io/ram.cpp.obj core/CMakeFiles/mrtrix-core.dir/image_io/scratch.cpp.obj core/CMakeFiles/mrtrix-core.dir/image_io/sparse.cpp.obj core/CMakeFiles/mrtrix-core.dir/image_io/variable_scaling.cpp.obj core/CMakeFiles/mrtrix-core.dir/math/SH.cpp.obj core/CMakeFiles/mrtrix-core.dir/math/average_space.cpp.obj core/CMakeFiles/mrtrix-core.dir/math/bessel.cpp.obj core/CMakeFiles/mrtrix-core.dir/math/betainc.cpp.obj core/CMakeFiles/mrtrix-core.dir/math/erfinv.cpp.obj core/CMakeFiles/mrtrix-core.dir/math/stats/fwe.cpp.obj core/CMakeFiles/mrtrix-core.dir/math/stats/glm.cpp.obj core/CMakeFiles/mrtrix-core.dir/math/stats/import.cpp.obj core/CMakeFiles/mrtrix-core.dir/math/stats/shuffle.cpp.obj core/CMakeFiles/mrtrix-core.dir/math/zstatistic.cpp.obj core/CMakeFiles/mrtrix-core.dir/misc/bitset.cpp.obj core/CMakeFiles/mrtrix-core.dir/mrtrix.cpp.obj core/CMakeFiles/mrtrix-core.dir/phase_encoding.cpp.obj core/CMakeFiles/mrtrix-core.dir/progressbar.cpp.obj core/CMakeFiles/mrtrix-core.dir/signal_handler.cpp.obj core/CMakeFiles/mrtrix-core.dir/stats.cpp.obj core/CMakeFiles/mrtrix-core.dir/stride.cpp.obj core/CMakeFiles/mrtrix-core.dir/thread.cpp.obj  C:/msys64/mingw64/lib/libpng.dll.a  C:/msys64/mingw64/lib/libtiff.dll.a  C:/msys64/mingw64/lib/libz.dll.a  C:/msys64/mingw64/lib/libfftw3.dll.a  C:/msys64/mingw64/lib/libfftw3f.dll.a  C:/msys64/mingw64/lib/libfftw3l.dll.a  core/libmrtrix-core-version-lib.a  C:/msys64/mingw64/lib/libfftw3.dll.a  C:/msys64/mingw64/lib/libfftw3f.dll.a  C:/msys64/mingw64/lib/libfftw3l.dll.a \
-lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 \
&& cd ."
daljit46 commented 6 months ago

Just to make sure I created a test repo to build mrtrix3 in debug mode here and it seems to work. So I'm inclined to think that there is something peculiar with your MSYS2 installation that's causing the problem. Could you send me the entire verbose build log (--verbose during build)? Teams or Pastebin would work.