Issue with Apple Silicon chip?

[Ceriz7]

Hi there,

I just freshly installed PGAP and tried to run it with the test genome on an M1 Macbook pro with macOS Big Sur. A warning/error then occurred as the following:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 'NoneType' object has no attribute 'group'

We wondered was there any success under Apple M1? As it is our first try in genome annotation we are not very sure if there is something wrong with our installation / operation or the platform used does not match well with the program?

Cheers!

[azat-badretdin]

Thank you for your report.

Were you able to get a successful result? Were you able to get a successful result for the sample genome we provided? Was this warning the extent of all the output?

As for Apple M1 - we do perform extensive testing under Linux in cloud environment and we test some other system modification (singularity, for example), but not on Apple.

As it is our first try in genome annotation we are not very sure if there is something wrong with our installation / operation or the platform used does not match well with the program?

Could you please provide more details on your run? Could you please post cwltool.log file?

Thanks!

[Ceriz7]

It is as following:

Last login: Thu Sep 23 11:03:46 on ttys000 ceri@YuweitekiMacBook-Pro ~ % ./pgap.py -r -o mg37_results test_genomes/MG37/input.yaml PGAP version 2021-07-01.build5508 is up to date. Output will be placed in: /Users/ceri/mg37_results.1 WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 'NoneType' object has no attribute 'group'

An output file is created afterward and the cwltool.log under it is like:

Original command: ./pgap.py -r -o mg37_results test_genomes/MG37/input.yaml

Docker command: /usr/local/bin/docker run -i --rm --user 501:20 --volume /Users/ceri/input-2021-07-01.build5508:/pgap/input:ro,z --volume /Users/ceri/test_genomes/MG37:/pgap/user_input:z --volume /Users/ceri/test_genomes/MG37/pgap_input_layymby1.yaml:/pgap/user_input/pgap_input.yaml:ro,z --volume /var/folders/3c/rmv5gsrd78qdr24qlgwvxgd80000gn/T/:/tmp:rw,z --volume /Users/ceri/mg37_results.1:/pgap/output:rw,z ncbi/pgap:2021-07-01.build5508 cwltool --timestamps --debug --disable-color --preserve-entire-environment --outdir /pgap/output pgap/pgap.cwl /pgap/user_input/pgap_input.yaml

--- Start YAML Input --- fasta: class: File location: ASM2732v1.annotation.nucleotide.1.fasta submol: class: File location: pgap_submol_7j2u2kx7.yaml supplemental_data: { class: Directory, location: /pgap/input } report_usage: true --- End YAML Input ---

--- Start Runtime Report ---

Cheers.

[azat-badretdin]

Do you have anything in mg37_results directory?

[Ceriz7]

Just a cwltool.log in the directory. The contents are as following:

Original command: ./pgap.py -r -o mg37_results test_genomes/MG37/input.yaml

Docker command: /usr/local/bin/docker run -i --rm --user 501:20 --volume /Users/ceri/input-2021-07-01.build5508:/pgap/input:ro,z --volume /Users/ceri/test_genomes/MG37:/pgap/user_input:z --volume /Users/ceri/test_genomes/MG37/pgap_input_layymby1.yaml:/pgap/user_input/pgap_input.yaml:ro,z --volume /var/folders/3c/rmv5gsrd78qdr24qlgwvxgd80000gn/T/:/tmp:rw,z --volume /Users/ceri/mg37_results.1:/pgap/output:rw,z ncbi/pgap:2021-07-01.build5508 cwltool --timestamps --debug --disable-color --preserve-entire-environment --outdir /pgap/output pgap/pgap.cwl /pgap/user_input/pgap_input.yaml

--- Start YAML Input --- fasta: class: File location: ASM2732v1.annotation.nucleotide.1.fasta submol: class: File location: pgap_submol_7j2u2kx7.yaml supplemental_data: { class: Directory, location: /pgap/input } report_usage: true --- End YAML Input ---

--- Start Runtime Report ---

[conmeehan]

Hi,

I am also having this problem. It is due to the built docker image requiring a linux/amd64 but the new Macs with M1 chips use linux/arm64/v8. The only way to correct this is to pass the --platform to the docker run command but I cant get pgap.py to do this. Is there a way to have the pipeline be useable on the M1 chips at all?

Thanks, Conor

[george-coulouris]

Unfortunately, we do not yet have native arm executables. While docker desktop on M1 theoretically can run amd64 images, doing so is very slow due to emulation, and is unsupported and untested at this time.

[rjavier17]

Hi.

Sorry to open this issue again, but I have exactly the same problem and I would like to know if it is already possible to run PGAP pipeline in a M1/M2/arm64 Apple Silicon based computer.

Thanks.

[azat-badretdin]

I am not aware of any changes in our software in that regard.

[francescodc87]

I am having the exact same problem. Did anybody found a solution?

[rjavier17]

Hi, Francesco.

Unfortunately I have been unable to make PGAP pipeline work on my Silicon Mac, Docker complains about wrong computer architecture. This is very inconvenient. I hope PGAP can be soon available for arm64 chips and/or for Anaconda/Conda, for instance.

[DanyMatute]

I am having the same problem, awaiting for the solution...

[azat-badretdin]

awaiting for the solution...

Unfortunately, the solution is quite far away from now. There is more than important factor that makes this quite difficult:

• large volume of code (only one gpipe component has more than a million lines of C++ code) to test and potentially port.
• most importantly, we have plenty of third party applications/binaries, failing to run even one of them on ARM platform will result in abortion of our pipeline

We have also tried emulation - that did not work for us either.

[francescodc87]

Did anybody find a solution or alternative for this?

[george-coulouris]

I was able to run pgap.py successfully using macOS 14.3 and Docker Desktop 4.27.1 (136059) using the workaround below.

The underlying executables are still not arm native, but performance is now acceptable under rosetta where previously it was not.

We will address this in a future release, but for now, you can try the following:

1. export DOCKER_DEFAULT_PLATFORM=linux/amd64
2. comment out the call to record_runtime() in pgap.py: # self.record_runtime(f)
3. use the --no-self-update option so that it doesn't overwrite your local changes to pgap.py.

[francescodc87]

Hi George, I tried your solution, but I am getting this error qemu: uncaught target signal 11 (Segmentation fault) - core dumped

When running this code ./pgap.py -r -o mg37_results -g $HOME/.pgap/test_genomes/MG37/ASM2732v1.annotation.nucleotide.1.fasta -s 'Mycoplasmoides genitalium' --no-self-update

am I doing something wrong?

[george-coulouris]

Hi @francescodc87! What version of the OS and docker are you running?

[francescodc87]

Hi @george-coulouris Docker version 25.0.3, build 4debf41 OS Sonoma 14.4