search

czimaginginstitute / MotionCor3

Anisotropic correction of beam induced sample motion for cryo-electron microscopy and tomography
BSD 3-Clause "New" or "Revised" License
41 stars 2 forks source link

Compiliation error with static binary library files (PIE object issues) #4

Closed charbj closed 4 months ago

charbj commented 8 months ago

Hi,

Having some issues with compilation. I've tried passing -fPIE and -no-pie to g++, but the error persists.

g++ and gcc versions are 9.4.0, trying to compile on Ubuntu 20.04.4 LTS. CUDA 11.6, nvcc V11.6.55.

/usr/bin/ld: /usr/local/programs/motioncor3/Lib/libmrcfile.a(CLoadExtHeader.o): relocation R_X86_64_32 against symbol __gxx_personality_v0@@CXXABI_1.3' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/local/programs/motioncor3/Lib/libmrcfile.a(CLoadImage.o): relocation R_X86_64_32 against symbol __gxx_personality_v0@@CXXABI_1.3' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/local/programs/motioncor3/Lib/libmrcfile.a(CLoadMrc.o): relocation R_X86_64_32 against symbol __gxx_personality_v0@@CXXABI_1.3' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/local/programs/motioncor3/Lib/libmrcfile.a(CSaveMrc.o): relocation R_X86_64_32 against symbol __gxx_personality_v0@@CXXABI_1.3' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/local/programs/motioncor3/Lib/libmrcfile.a(CSaveMainHeader.o): relocation R_X86_64_32 against symbol __gxx_personality_v0@@CXXABI_1.3' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/local/programs/motioncor3/Lib/libutil.a(Util_LinEqs.o): relocation R_X86_64_32 against '.rodata' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/local/programs/motioncor3/Lib/libutil.a(Util_Thread.o): relocation R_X86_64_32 against symbol '_ZTV11Util_Thread' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/local/programs/motioncor3/Lib/libutil.a(Util_FileName.o): relocation R_X86_64_32 against '.rodata' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/local/programs/motioncor3/Lib/libutil.a(Util_String.o): relocation R_X86_64_32 against '.rodata' can not be used when making a PIE object; recompile with -fPIE collect2: error: ld returned 1 exit status make: *** [makefile11:160: exe] Error 1

Thanks. CBJ

homurachan commented 8 months ago

Same problem here. It seems those three ".a" files in "MotionCor3/Lib" need to be recompiled by the author.

dirrei commented 8 months ago

Same issue here, on Ubuntu 20.04 with Cuda 11.8 .

k-haru commented 8 months ago

The following changes worked for me but have not been tested. makefile11

exe: $(OBJS)
        @$(NVCC) -g -G -m64 $(OBJS) \
        $(PRJLIB)/libmrcfile.a $(PRJLIB)/libutil.a $(PRJLIB)/libcuutil.a \
        -L$(CUDALIB) \
        -L$(CONDA)/lib \
        -L/usr/lib64 \
        -Xcompiler -no-pie \ 
        -lcufft -lcudart -lcuda -lnvToolsExt -ltiff -lc -lm -lpthread \
        -o MotionCor3
        @echo MotionCor3 has been generated.
homurachan commented 8 months ago

The following changes worked for me but have not been tested. makefile11

exe: $(OBJS)
        @$(NVCC) -g -G -m64 $(OBJS) \
        $(PRJLIB)/libmrcfile.a $(PRJLIB)/libutil.a $(PRJLIB)/libcuutil.a \
        -L$(CUDALIB) \
        -L$(CONDA)/lib \
        -L/usr/lib64 \
        -Xcompiler -no-pie \ 
        -lcufft -lcudart -lcuda -lnvToolsExt -ltiff -lc -lm -lpthread \
        -o MotionCor3
        @echo MotionCor3 has been generated.

This really works. After compilation I tested the performances and results, all looking good.

yanliu-ss commented 8 months ago

I have the same problem but adding ' -Xcompiler -no-pie 'didn't work. I'm on Ubuntu 20.04.4, CUDA 12.1. Any suggestions?

elarquet commented 8 months ago

I have the same problem. Adding ' -Xcompiler -no-pie ' or ' - Xlinker -no-pie ' didn't work (the compilation is carried out without error but the execution of the program gives rise to a segmentation fault). I'm on Ubuntu 22.04, CUDA 12.3. It seems those three ".a" files in "MotionCor3/Lib" need to be recompiled by the author with -fPIE flag.

MTclement1 commented 8 months ago

I have the same problem. Adding ' -Xcompiler -no-pie ' or ' - Xlinker -no-pie ' didn't work (the compilation is carried out without error but the execution of the program gives rise to a segmentation fault). I'm on Ubuntu 22.04, CUDA 12.3. It seems those three ".a" files in "MotionCor3/Lib" need to be recompiled by the author with -fPIE flag.

I had the segfault only when using -Serial 1 for batch. Does it work for you on single file ?

szhengczii commented 8 months ago

We will try to release the source of these .a files as soon as we can.

elarquet commented 8 months ago

I have the same problem. Adding ' -Xcompiler -no-pie ' or ' - Xlinker -no-pie ' didn't work (the compilation is carried out without error but the execution of the program gives rise to a segmentation fault). I'm on Ubuntu 22.04, CUDA 12.3. It seems those three ".a" files in "MotionCor3/Lib" need to be recompiled by the author with -fPIE flag.

I had the segfault only when using -Serial 1 for batch. Does it work for you on single file

Exactly the same issue, work well for a single file, but segmentation fault for -serial 1 option

nayimgr commented 8 months ago

I have the same problem. Adding ' -Xcompiler -no-pie ' or ' - Xlinker -no-pie ' didn't work (the compilation is carried out without error but the execution of the program gives rise to a segmentation fault). I'm on Ubuntu 22.04, CUDA 12.3. It seems those three ".a" files in "MotionCor3/Lib" need to be recompiled by the author with -fPIE flag.

Same problem as above, same version of Ubuntu and CUDA 12.0. The suggested fix didn't work for us.

elarquet commented 5 months ago

We will try to release the source of these .a files as soon as we can.

Hi, Some news concerning the source of /Lib *.a files ? Thanks,

TimKirk commented 5 months ago

For what it's worth the

-Xcompiler -no-pie \

fix worked for me just now on Ubuntu 20.04, Cuda 12.3

elarquet commented 5 months ago

For what it's worth the

-Xcompiler -no-pie \

fix worked for me just now on Ubuntu 20.04, Cuda 12.3

Could you confirm that it worked both on single file and using -Serial 1 for batch !

TimKirk commented 4 months ago

For what it's worth the -Xcompiler -no-pie \ fix worked for me just now on Ubuntu 20.04, Cuda 12.3

Could you confirm that it worked both on single file and using -Serial 1 for batch !

Sorry for the slow response - I just run the computers here so had to get someone with data to test the apparently successful build actually worked properly.

I was told it seemed to be working fine : "Serial mode is slower than RELION's implementation (using 4 MPI and 12 threads). RELION used to run multiple MotionCor2 processes in parallel and each isn't using all GPU memory so I tried that and that is faster than RelionCor."

I hope that's what you were wanting to know. If there are more tests I can run without need my own datasets, let me know.

elarquet commented 4 months ago

For what it's worth the -Xcompiler -no-pie fix worked for me just now on Ubuntu 20.04, Cuda 12.3

Could you confirm that it worked both on single file and using -Serial 1 for batch !

Sorry for the slow response - I just run the computers here so had to get someone with data to test the apparently successful build actually worked properly.

I was told it seemed to be working fine : "Serial mode is slower than RELION's implementation (using 4 MPI and 12 threads). RELION used to run multiple MotionCor2 processes in parallel and each isn't using all GPU memory so I tried that and that is faster than RelionCor."

I hope that's what you were wanting to know. If there are more tests I can run without need my own datasets, let me know.

Thanks Tim for this information.

elarquet commented 4 months ago

We will try to release the source of these .a files as soon as we can.

I found a possible explanation regarding the -fPIE compile flag issue: https://www.mjr19.org.uk/IT/pic_pie.html

szhengczii commented 4 months ago

Dear colleagues,

We are going to release MotionCor3 1.1.0 this week that has the source code of .a files. Thanks for helping resolve this issue.

Best Shawn

On Mon, Feb 19, 2024 at 1:42 AM elarquet @.***> wrote:

We will try to release the source of these .a files as soon as we can.

I found a possible explanation regarding the -fPIE compile flag issue: https://www.mjr19.org.uk/IT/pic_pie.html

— Reply to this email directly, view it on GitHub https://github.com/czimaginginstitute/MotionCor3/issues/4#issuecomment-1952062755, or unsubscribe https://github.com/notifications/unsubscribe-auth/BBUDUPZVGRKAG4PW3YULRODYUMNBJAVCNFSM6AAAAAA6VHPJUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJSGA3DENZVGU . You are receiving this because you commented.Message ID: @.***>

lucajovine commented 4 months ago

Hi, I'm still getting the same issue when trying to compile MotionCor3 1.1.1 which you just announced... any way around this? Thanks, Luca

lucajovine commented 4 months ago

Ah OK, it DOES work if one goes into LibSrc/Mrcfile and LibSrc/Util and recompiles the libraries. I wrongly assumed this was done automatically. Anyway - problem solved and thank you for posting the new version!

elarquet commented 4 months ago

Dear colleagues, We are going to release MotionCor3 1.1.0 this week that has the source code of .a files. Thanks for helping resolve this issue. Best Shawn

Many Thanks Shawn! It works perfectly now after reconstructing Libraries with the -fPIE Flag. May I make one final request of you? Can you make available the sources of the libcuutilfft.a and libcuutil.a libraries as you did for libmrcfile.a and libutil.a in the case of the GctfFind software. Thanks again, kind regards, Eric.

szhengczii commented 4 months ago

The next release of GCtfFind, similar to MotionCor3 and AreTomo2, will merge the needed code in libcuutilfft.a and libcuutil.a and provide the source code of libutil and libmrcfile. You will have everything you need to compile it. However, it will take a while for me to have time working on it. Thanks for your patience and my apologies for the long delay.

Best, Shawn

On Sun, Mar 3, 2024 at 3:40 AM elarquet @.***> wrote:

Dear colleagues, We are going to release MotionCor3 1.1.0 this week that has the source code of .a files. Thanks for helping resolve this issue. Best Shawn

Many Thanks Shawn! It works perfectly now after reconstructing Libraries with the -fPIE Flag. May I make one final request of you? Can you make available the sources of the libcuutilfft.a and libcuutil.a libraries as you did for libmrcfile.a and libutil.a in the case of the GctfFind software. Thanks again, kind regards, Eric.

— Reply to this email directly, view it on GitHub https://github.com/czimaginginstitute/MotionCor3/issues/4#issuecomment-1975131704, or unsubscribe https://github.com/notifications/unsubscribe-auth/BBUDUPZSV2TOXGRQJYZMS6LYWMD4HAVCNFSM6AAAAAA6VHPJUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZVGEZTCNZQGQ . You are receiving this because you commented.Message ID: @.***>

szhengczii commented 4 months ago

MotionCor3 has provided the source code of these libraries. Recompile the libraries in LibSrc should fix this problem.