search

pangenome / pggb

the pangenome graph builder
https://doi.org/10.1038/s41592-024-02430-3
MIT License
368 stars 41 forks source link

docker build error with MacBook Pro #137

Closed zmz1988 closed 3 years ago

zmz1988 commented 3 years ago

Hi! I trying to build an image following the instruction, with command line docker build --target binary -t ${USER}/pggb:latest . in pggb directory, but I get an error as following:

` ...

8 40.71 [ 85%] Building CXX object CMakeFiles/seqwish.dir/src/main.cpp.o

8 40.71 [ 85%] Building CXX object CMakeFiles/seqwish.dir/src/utils.cpp.o

8 40.71 [ 86%] Building CXX object CMakeFiles/seqwish.dir/src/seqindex.cpp.o

8 40.71 c++: error: unrecognized command-line option '-mcx16'

8 40.71 c++: error: unrecognized command-line option '-mcx16'

8 40.71 gmake[2]: *** [CMakeFiles/seqwish.dir/build.make:82: CMakeFiles/seqwish.dir/src/utils.cpp.o] Error 1

8 40.71 gmake[2]: *** Waiting for unfinished jobs....

8 40.71 gmake[2]: *** [CMakeFiles/seqwish.dir/build.make:95: CMakeFiles/seqwish.dir/src/main.cpp.o] Error 1

8 40.71 c++: error: unrecognized command-line option '-mcx16'

8 40.71 gmake[2]: *** [CMakeFiles/seqwish.dir/build.make:108: CMakeFiles/seqwish.dir/src/seqindex.cpp.o] Error 1

8 40.71 [ 87%] Building CXX object CMakeFiles/seqwish.dir/src/paf.cpp.o

8 40.72 c++: error: unrecognized command-line option '-mcx16'

8 40.72 gmake[2]: *** [CMakeFiles/seqwish.dir/build.make:121: CMakeFiles/seqwish.dir/src/paf.cpp.o] Error 1

8 40.72 gmake[1]: *** [CMakeFiles/Makefile2:454: CMakeFiles/seqwish.dir/all] Error 2

8 40.72 gmake: *** [Makefile:149: all] Error 2

executor failed running [/bin/sh -c git clone --recursive https://github.com/ekg/seqwish && cd seqwish && git pull && git checkout 6da2102 && git submodule update --init --recursive && cmake -H. -Bbuild && cmake --build build -- -j $(nproc) && cp bin/seqwish /usr/local/bin/seqwish && cd ../]: exit code: 2 `

Could you please help me with this problem? Thanks a lot in advance!

ekg commented 3 years ago

It looks like your GCC (in docker?) Doesn't have the -mcx16 flag.

On Mon, Sep 20, 2021, 21:57 zzz @.***> wrote:

Hi! I trying to build an image following the instruction, with command line docker build --target binary -t ${USER}/pggb:latest . in pggb directory, but I get an error as following: ` ...

8 https://github.com/pangenome/pggb/pull/8 40.71 [ 85%] Building CXX

object CMakeFiles/seqwish.dir/src/main.cpp.o

8 https://github.com/pangenome/pggb/pull/8 40.71 [ 85%] Building CXX

object CMakeFiles/seqwish.dir/src/utils.cpp.o

8 https://github.com/pangenome/pggb/pull/8 40.71 [ 86%] Building CXX

object CMakeFiles/seqwish.dir/src/seqindex.cpp.o

8 https://github.com/pangenome/pggb/pull/8 40.71 c++: error:

unrecognized command-line option '-mcx16'

8 https://github.com/pangenome/pggb/pull/8 40.71 c++: error:

unrecognized command-line option '-mcx16'

8 https://github.com/pangenome/pggb/pull/8 40.71 gmake[2]: ***

[CMakeFiles/seqwish.dir/build.make:82: CMakeFiles/seqwish.dir/src/utils.cpp.o] Error 1

8 https://github.com/pangenome/pggb/pull/8 40.71 gmake[2]: *** Waiting

for unfinished jobs....

8 https://github.com/pangenome/pggb/pull/8 40.71 gmake[2]: ***

[CMakeFiles/seqwish.dir/build.make:95: CMakeFiles/seqwish.dir/src/main.cpp.o] Error 1

8 https://github.com/pangenome/pggb/pull/8 40.71 c++: error:

unrecognized command-line option '-mcx16'

8 https://github.com/pangenome/pggb/pull/8 40.71 gmake[2]: ***

[CMakeFiles/seqwish.dir/build.make:108: CMakeFiles/seqwish.dir/src/seqindex.cpp.o] Error 1

8 https://github.com/pangenome/pggb/pull/8 40.71 [ 87%] Building CXX

object CMakeFiles/seqwish.dir/src/paf.cpp.o

8 https://github.com/pangenome/pggb/pull/8 40.72 c++: error:

unrecognized command-line option '-mcx16'

8 https://github.com/pangenome/pggb/pull/8 40.72 gmake[2]: ***

[CMakeFiles/seqwish.dir/build.make:121: CMakeFiles/seqwish.dir/src/paf.cpp.o] Error 1

8 https://github.com/pangenome/pggb/pull/8 40.72 gmake[1]: ***

[CMakeFiles/Makefile2:454: CMakeFiles/seqwish.dir/all] Error 2

8 https://github.com/pangenome/pggb/pull/8 40.72 gmake: ***

[Makefile:149: all] Error 2

executor failed running [/bin/sh -c git clone --recursive https://github.com/ekg/seqwish && cd seqwish && git pull && git checkout 6da2102 && git submodule update --init --recursive && cmake -H. -Bbuild && cmake --build build -- -j $(nproc) && cp bin/seqwish /usr/local/bin/seqwish && cd ../]: exit code: 2 `

Could you please help me with this problem? Thanks a lot in advance!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pangenome/pggb/issues/137, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABDQEOQVJEKNC2ABRLKV3DUC6G4FANCNFSM5EMWIS4Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

subwaystation commented 3 years ago

Hi @zmz1988,

for me docker build --target binary -t ${USER}/pggb:latest . works with the latest pggb commit. The Dockerfile itself is designed in a way to already have the GCC version incorporated that's required to compile seqwish. So my impression is that you maybe changed the Dockerfile? Did you just go for

git clone --recursive https://github.com/pangenome/pggb
cd pggb
docker build --target binary -t ${USER}/pggb:latest .

? Or it could be your hardware does not support -mcx16. But that seems very unlikely: https://superuser.com/questions/187254/how-prevalent-are-old-x64-processors-lacking-the-cmpxchg16b-instruction.

zmz1988 commented 3 years ago

Hi @ekg @subwaystation , Thank you both for answering me!

I did exactly as suggested in the installation section of readme file. I never use docker before, so I had to install docker with the version of Mac with Apple chip. Then in my terminal, use the exact lines as showed above. I'm still so naive to understand how to change the docker file...

I checked my machine. After gcc --version: Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1 Apple clang version 12.0.5 (clang-1205.0.22.11) Target: arm64-apple-darwin20.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin

I also checked the options for the gcc command by typing gcc- help in terminal, and I didn't find -mcx16. So I guess that my machine does not support -mcx16?

I tried to pull a gcc version in docker, but am still slowly learning how to do it...

subwaystation commented 3 years ago

Your local GCC should not affect the GCC when running the pggb docker container. Can you please try docker pull ghcr.io/pangenome/pggb:latest followed by docker run -it -v ${PWD}/data/:/data ghcr.io/pangenome/pggb:latest "pggb -i /data/HLA/DRB1-3123.fa.gz -N -s 5000 -I 0 -p 80 -n 10 -k 8 -t 2 -v -L -o /data/out -m"? First you pull the already created docker image, then you run it with a test data set. Make sure that you cd into the pggb folder first!

zmz1988 commented 3 years ago

I get an error. Please see the following:

MacBook-Pro-2 pggb % docker run -it -v ${PWD}/data/:/data ghcr.io/pangenome/pggb:latest "pggb -i /data/HLA/DRB1-3123.fa.gz -N -s 5000 -I 0 -p 80 -n 10 -k 8 -t 2 -v -L -o /data/out -m" WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested [wfmash::map] Reference = [/data/HLA/DRB1-3123.fa.gz] [wfmash::map] Query = [/data/HLA/DRB1-3123.fa.gz] [wfmash::map] Kmer size = 16 [wfmash::map] Window size = 2 [wfmash::map] Segment length = 5000 (read split disabled) [wfmash::map] Block length min = 15000 [wfmash::map] Alphabet = DNA [wfmash::map] Percentage identity threshold = 80% [wfmash::map] Mapping output file = //wfmash-s3ZAf5 [wfmash::map] Filter mode = 1 (1 = map, 2 = one-to-one, 3 = none) [wfmash::map] Execution threads = 2 [wfmash::skch::Sketch::build] minimizers picked from reference = 108363 terminate called after throwing an instance of 'std::overflow_error' what(): robin_hood::map overflow qemu: uncaught target signal 6 (Aborted) - core dumped Command terminated by signal 6 wfmash -X -s 5000 -l 15000 -N -p 80 -n 10 -k 16 -t 2 /data/HLA/DRB1-3123.fa.gz /data/HLA/DRB1-3123.fa.gz 0.34s user 0.01s system 101% cpu 0.35s total 22548Kb max memory

Is it related the memory issue that I don't have enough memory? I had to run it locally, as the cluster I use does not support docker...

subwaystation commented 3 years ago

Alright, now I get the problem. You have an ARM64 CPU architecture, but all our tools are build for the AMD64 architecture. No wonder you couldn't build seqwish. It was because an instruction specifically for the AMD64 architecture was set.

Maybe a better way to approach this is tool by tool, first. Because we didn't test any of them on a ARM64 CPU architecture. I am afraid, non of the major developers has access to such a machine. Get a more recent GCC and try to build https://github.com/ekg/wfmash, https://github.com/pangenome/smoothxg as instructed in the README. If that works we can take a closer look at the seqwish problem. But it might take some time to figure out some compiler errors. Unfortunately, you can't install all the tools from Bioconda, because the ARM64 support is still pretty scarce there, too.

@ekg @AndreaGuarracino What do you think about ARM64 regarding pggb and its tools?

ekg commented 3 years ago

We should support arm64!

On Tue, Sep 21, 2021 at 4:26 PM Simon Heumos @.***> wrote:

Alright, now I get the problem. You have an ARM64 CPU architecture, but all our tools are build for the AMD64 architecture. No wonder you couldn't build seqwish. It was because an instruction specifically for the AMD64 architecture was set.

Maybe a better way to approach this is tool by tool, first. Because we didn't test any of them on a ARM64 CPU architecture. I am afraid, non of the major developers has access to such a machine. Get a more recent GCC and try to build https://github.com/ekg/wfmash, https://github.com/pangenome/smoothxg as instructed in the README. If that works we can take a closer look at the seqwish problem. But it might take some time to figure out some compiler errors. Unfortunately, you can't install all the tools from Bioconda, because the ARM64 support is still pretty scarce there, too.

@ekg https://github.com/ekg @AndreaGuarracino https://github.com/AndreaGuarracino What do you think about ARM64 regarding pggb and its tools?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pangenome/pggb/issues/137#issuecomment-924042546, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABDQELZOYVAH2SL3URADV3UDCIZPANCNFSM5EMWIS4Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

zmz1988 commented 3 years ago

Oh I didn't take the warning message so seriously, when I tested the test file! Now I know the problem! I will change to a computer with Intel core and try!

Thanks a lot! Also looking forward to the version that can support arm64!

subwaystation commented 3 years ago

We should support arm64!

Seems like Rust won't be there any time soon! https://github.com/rust-lang/rust/issues/73908#issuecomment-798814937.

subwaystation commented 3 years ago

@zmz1988 Please tell us, if switching the machine architecture worked for you :)

zmz1988 commented 3 years ago

@subwaystation Works smoothly in MacBook with Intel core i7! Thanks a lot!!

subwaystation commented 3 years ago

Thanks for letting us know! Have fun building graphs ;)

The support of arm64 is not on a very close horizon, so I am closing this issue.