search

chanzuckerberg / miniwdl

Workflow Description Language developer tools & local runner
MIT License
173 stars 54 forks source link

Miniwdl does not work on ARM macs (e.g. M1) #652

Open corneliusroemer opened 1 year ago

corneliusroemer commented 1 year ago

I've just hit this issue previously mentioned in a comment and would like to increase visibility by making it an issue of its own.

Would be great if the README could mention that ARM macs are currently not supported to reduce time waste trying things out.

This is @eholdmore's comment:

Hi @mlin and others, thanks for all the hard work here. I just wanted to add that miniwdl run_self_test fails on macOS Venture 13.1 with M1 for me. Here are the steps I followed:

  1. Install Docker Desktop and ensure Docker is running with docker ps
  2. Set file sharing to osxfs (Legacy) as suggested by https://github.com/chanzuckerberg/miniwdl/issues/461#issuecomment-1142741511
  3. Install Miniconda3
  4. Install miniwdl using conda install -c conda-forge miniwdl
  5. export TMPDIR=/tmp
  6. Finally, miniwdl run_self_test

System Details macOS: Ventura 13.2 Chip: M1 Docker: 4.16.2 Python: 3.10.8 MiniWDL: 0.10.0

Output ~ % miniwdl run_self_test test.wdl workflow hello_caller scatter name call hello if task hello

Originally posted by @eholdmore in https://github.com/chanzuckerberg/miniwdl/issues/145#issuecomment-1410898132

mlin commented 1 year ago

@corneliusroemer Thanks for raising this, indeed in the first couple years of Apple Silicon, I don't think anyone had much expectations for Docker-based bioinformatics workflows to work on that platform, so our docs did not address it -- but I presume that's changing by now, and you're right to point it out. I have made some updates to the readme and the pinned issue #145, also linking here to continue discussion.

miniwdl itself is purely Python so I don't think it has any fundamental incompatibility with ARM; the questions would be (1) is the right version of everything getting installed and (2) is it able to talk to Docker and use the applicable Docker images. Unfortunately, I'm still using an Intel MacBook and GitHub Actions still only has Intel macOS runners. If somebody with a newer Mac were to hack on it, I doubt it would be heavy lift to get it working so long as the desired Docker images can be run. And the non-runner tools like miniwdl check ought to work, but I'd be curious if anyone can confirm.

The backtrace posted by @eholdmore bears on question 1 because it curiously seems to refer to an issue with a version of miniwdl that was already outdated at the time. The 'EntryPoints' object has no attribute 'get', caused by an unexpected change in a dependent package, had been fixed last October. So I wonder if our conda channel is somehow distributing an older code revision to ARM macs specifically. Is that error among the ones you encountered too?

adamnovak commented 1 year ago

I'm able to run MiniWDL on an M1 Mac.

I installed MiniWDL from pip into a virtual environment, something like:

pip3 install virtualenv
virtualenv venv
. venv/bin/activate
pip install miniwdl

Then I need to export DOCKER_HOST=unix://${HOME}/.docker/run/docker.sock (because that's where Docker Desktop puts its socket), and make sure Docker Desktop is running and has "VirtioFS" set as its file sharing mode, and then I can miniwdl run_self_test just fine.

mschatz commented 1 year ago

@adamnovak I was unable to get this to work. miniwdl run_self_test gets stuck in a loop: 

...
2023-09-19 22:01:11.456 wdl.w:hello_caller.download0.t:download-aria2c waiting for local docker swarm manager & worker(s) :: manager: "error", workers: 0
2023-09-19 22:01:13.498 urllib3.connectionpool http://localhost:None "GET /v1.43/info HTTP/1.1" 200 None
2023-09-19 22:01:13.503 wdl.w:hello_caller.download0.t:download-aria2c swarm info :: NodeID: "", NodeAddr: "127.0.0.1", LocalNodeState: "error", ControlAvailable: false, Error: "error while loading TLS certificate in /var/lib/docker/swarm/certificates/swarm-node.crt: certificate (1 - ogip74kecsovzofirtkrm1jqj) not valid after Sat, 24 Dec 2022 23:55:00 UTC, and it is currently Wed, 20 Sep 2023 00:09:06 UTC: x509: certificate has expired or is not yet valid: ", RemoteManagers: null
2023-09-19 22:01:13.505 wdl.w:hello_caller.download0.t:download-aria2c waiting for local docker swarm manager & worker(s) :: manager: "error", workers: 0
2023-09-19 22:01:15.550 urllib3.connectionpool http://localhost:None "GET /v1.43/info HTTP/1.1" 200 None
2023-09-19 22:01:15.551 wdl.w:hello_caller.download0.t:download-aria2c swarm info :: NodeID: "", NodeAddr: "127.0.0.1", LocalNodeState: "error", ControlAvailable: false, Error: "error while loading TLS certificate in /var/lib/docker/swarm/certificates/swarm-node.crt: certificate (1 - ogip74kecsovzofirtkrm1jqj) not valid after Sat, 24 Dec 2022 23:55:00 UTC, and it is currently Wed, 20 Sep 2023 00:09:06 UTC: x509: certificate has expired or is not yet valid: ", RemoteManagers: null
2023-09-19 22:01:15.552 wdl.w:hello_caller.download0.t:download-aria2c waiting for local docker swarm manager & worker(s) :: manager: "error", workers: 0

Any tips on what else to try?

Mike

adamnovak commented 1 year ago

@mschatz It looks like your Docker daemon is set up to be in a swarm (which I think MiniWDL needs), but the swarm's root certificate has expired.

I think you should be able to fix it by destroying the VM image that Docker for Mac is using, and making it create a new one and make a new certificate. I think you can do that by going to the Docker tray menu, hitting "Troubleshoot", and then hitting "Clean / Purge Data".

You might also be able to do docker swarm ca --rotate?

When I do:

docker swarm ca >ca.pem
openssl x509 -in ca.pem -noout -text

it shows me that I am using a certificate that expires in 2043. So I think the certificates used to get a 1 year lifetime and now they get a 20 year lifetime by default.

mschatz commented 12 months ago

Thanks for the tip - you were exactly right! Mike

On Thu, Sep 21, 2023 at 11:19 AM Adam Novak @.***> wrote:

@mschatz https://github.com/mschatz It looks like your Docker daemon is set up to be in a swarm (which I think MiniWDL needs), but the swarm's root certificate has expired.

I think you should be able to fix it by destroying the VM image that Docker for Mac is using, and making it create a new one and make a new certificate. I think you can do that by going to the Docker tray menu, hitting "Troubleshoot", and then hitting "Clean / Purge Data".

You might also be able to do docker swarm ca --rotate?

When I do:

docker swarm ca >ca.pem openssl x509 -in ca.pem -noout -text

it shows me that I am using a certificate that expires in 2043. So I think the certificates used to get a 1 year lifetime and now they get a 20 year lifetime by default.

— Reply to this email directly, view it on GitHub https://github.com/chanzuckerberg/miniwdl/issues/652#issuecomment-1729798219, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABP34ZXLBJX2A4P4EIGX6DX3RLG3ANCNFSM6AAAAAA3BC3SNQ . You are receiving this because you were mentioned.Message ID: @.***>

watronfire commented 9 months ago

I had a similar issue on my Macbook with an M2. I installed docker, set the file sharing to osxfs, and installed miniwdl with mamba. run_self_test failed with the same error. But it turns out that conda installed v0.10.0 rather than the current version (at this moment v1.11.1). As @mlin said, I think v0.10.0 is the latest version of miniwdl conda-forge has, whereas for intel macs it has v1.11.1.

Because of this I was able to successfully run the self test after uninstalling miniwdl and reinstalling by temporarily setting my conda subdirectory to osx-64, i.e. CONDA_SUBDIR=osx-64 mamba install miniwdl