search

chanzuckerberg / miniwdl

Workflow Description Language developer tools & local runner
MIT License
175 stars 54 forks source link

macOS compatibility #145

Open mlin opened 5 years ago

mlin commented 5 years ago

miniwdl is tested on Intel macOS, with a few specific setup steps:

Then, proceed through the Getting Started tutorial if so desired.

miniwdl is NOT officially supported on Apple Silicon; but may work experimentally, depending on the required task container images. Further discussion #652

To set up for miniwdl development:

conda create miniwdl-dev pip jq coreutils
conda activate miniwdl-dev
pip3 install -r requirements.dev.txt
make qtest

(make sure which stat refers to coreutils' version)

illusional commented 5 years ago

Hey @mlin, I'm trying out WDL and having troubles using the run functionality on macOS when running an untar workflow. (I thought i'd drop it here in case it relates to macOS). I'm getting the error:

franklinmichael$ miniwdl run simple.wdl -i inp-local.json 
WARNING:wdl-workflow:simple:starting workflow simple (simple.wdl Ln 7 Col 1) in /Users/franklinmichael/Desktop/workflows-for-testing/03-simple/wdl/20190722_073803_simple
WARNING:wdl-worfklow:simple:issue call-untar on untar
ERROR:wdl-task:call-untar:failed to close docker-py client
Traceback (most recent call last):
  File "/anaconda3/lib/python3.7/site-packages/WDL/runtime/task.py", line 296, in _run
    assert exit_info
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda3/lib/python3.7/site-packages/WDL/runtime/task.py", line 306, in _run
    client.close()
  File "/anaconda3/lib/python3.7/site-packages/docker/client.py", line 187, in __getattr__
    raise AttributeError(' '.join(s))
AttributeError: 'DockerClient' object has no attribute 'close' In Docker SDK for Python 2.0, this method is now on the object APIClient. See the low-level API section of the documentation for more details.
ERROR:wdl-task:call-untar:task untar (tools/untar.wdl Ln 3 Col 1) failed: AssertionError
ERROR:wdl-workflow:simple:call-untar failed
ERROR:miniwdl-run:AssertionError,

I presume this might be related to needing permission to control Docker.

The invoking user must have permission to control Docker.

I've already changed my user-specific temp directory for mixing CWLTool and Docker, and I'm running miniwdl 0.3.0, Docker Desktop community 2.0.5.0, and the Docker python module 2.5.1.

mlin commented 5 years ago

@illusional based on those tracebacks I suspect the installed docker-py package might need updating (pip3 install --upgrade docker or such). Can you see if that makes a difference?

(More broadly, we don't have routine testing on macOS yet, need to plow through the issues above and get the test suite going on Travis)

illusional commented 5 years ago

@mlin Amazing, I didn't realise how out of date my Docker was (due to a Toil dependency). miniwdl run works on my small workflow. Will try on some bigger workflows and let you know how to goes.

I'm not sure what the min version of Docker this would require, but might be worth adding a minimum in setuptools install_requires.

mlin commented 5 years ago

Dang, it seems that Travis' macOS environment doesn't support Docker for Mac, so we're going to have to come up with some other approach for Mac testing.

illusional commented 5 years ago

Hey @mlin, just wanted to report back and say that I ran miniwdl on a germline processing pipeline (3 variant callers, scattering, gathering and small expressions) and my outputs were identical to Cromwell. This is totally awesome!

mlin commented 5 years ago

@illusional thank you for the field report; much appreciated! @mckinsel @bkmartinjr @tjchen

mlin commented 3 years ago

I've been seeing flakiness with Docker for Mac's volume mounting. Sporadically (maybe less than 1% of container starts) the WDL input files appear to be empty (mount point exists, but zero length) inside the container, with no errors reported by Docker. This is quite concerning because zero-length inputs might be valid for some tasks (e.g., line count) so there's no obvious way to know to retry them.

The way Docker for Mac mounts host volumes into containers (in a VM) is pretty complicated.

cc @lynnlangit

tomkinsc commented 3 years ago

I encountered the task-input-is-empty issue repeatedly (deterministically?) tonight when trying to run a wdl workflow on macOS. That gave me a chance to turn some knobs. A temporary solution—for me at least— is to disable the Use gRPC FUSE for file sharing toggle in the preferences of the macOS docker VM host (screenshots of the working state attached, along with my docker versions). This is on macOS Catalina 10.15.6 w/ Docker Desktop 2.5.0.0 and Docker Engine 19.03.13. I didn't want to delve further into FUSE debugging this evening, but wanted to mention this here in case it is something to go on. If the issues on docker/for-mac are any indication, this may be more of a docker problem than a miniwdl problem.

Edit: The problem is still present after updating to Docker Desktop 2.5.0.1 if Use gRPC FUSE for file sharing is enabled (disabling it still resolves the issue).

Screen Shot 2020-11-24 at 9 44 50 PM Screen Shot 2020-11-24 at 9 47 55 PM

markjschreiber commented 3 years ago

What is the expected pyre version for running make test. I have pyre 0.9.8 in the virtual env but when I run make test I see the following which suggests that the --show-parse-errors flag doesn't exist:

# regression test against pyre doing nothing (issue #100)
echo "check_check: str = 42" > WDL/DELETEME_check_check.py
/Applications/Xcode.app/Contents/Developer/usr/bin/make check > /dev/null 2>&1 && exit 1 || exit 0
rm WDL/DELETEME_check_check.py
pyre \
                --search-path stubs \
                --typeshed `python3 -c 'import sys, site, os; print(next(p for p in (os.path.join(dir,"lib/pyre_check/typeshed") for dir in (sys.prefix,site.getuserbase())) if os.path.isdir(p)))'` \
                --show-parse-errors check
Usage: pyre [OPTIONS] COMMAND [ARGS]...
Try 'pyre -h' for help.

Error: No such option: --show-parse-errors (Possible options: --no-show-error-traces, --show-error-traces)
make: *** [check] Error 2
vsmalladi commented 2 years ago

Have the issue that inputs are not found.

This is on macOS Monterey 12.0.1 w/ Docker Desktop 4.2.0 and Docker Engine 20.10.10. Toggled Use gRPC FUSE for file sharing off. Any help would be appreciated?

aofarrel commented 2 years ago

Minor update needed on the original post here. "Use gRPC FUSE for file sharing" is no longer under Docker's experimental features, it's in general.

mschatz commented 2 years ago

FYI - There is a currently a very weird bug in Docker desktop 4.12.0 (docker engine 20.10.17) on Apple M1 where if you try to disable "Use gRPC FUSE for file sharing" via the GUI it will turn itself back on when you try to activate it. Docker is aware of the problem and are working on a fix. In the meantime there is a workaround available: https://github.com/docker/for-mac/issues/6467

Mike

eholdmore commented 1 year ago

Hi @mlin and others, thanks for all the hard work here. I just wanted to add that miniwdl run_self_test fails on macOS Venture 13.1 with M1 for me. Here are the steps I followed:

  1. Install Docker Desktop and ensure Docker is running with docker ps
  2. Set file sharing to osxfs (Legacy) as suggested by https://github.com/chanzuckerberg/miniwdl/issues/461#issuecomment-1142741511
  3. Install Miniconda3
  4. Install miniwdl using conda install -c conda-forge miniwdl
  5. export TMPDIR=/tmp
  6. Finally, miniwdl run_self_test

System Details macOS: Ventura 13.2 Chip: M1 Docker: 4.16.2 Python: 3.10.8 MiniWDL: 0.10.0

Output ~ % miniwdl run_self_test test.wdl workflow hello_caller scatter name call hello if task hello