New install error

[iamdamion]

This used to work great on my old lab computer. I just got new one that is running mac os mojave (I'm guessing this is the issue?) but with the same conda env I've always used. I now get this error:

  0%|          | 0/2331 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/Users/d/miniconda2/envs/fmri/bin/dicomsorter", line 10, in <module>
    sys.exit(main())
  File "/Users/d/miniconda2/envs/fmri/lib/python2.7/site-packages/dicomsorter/cli.py", line 146, in main
    run()
  File "/Users/d/miniconda2/envs/fmri/lib/python2.7/site-packages/dicomsorter/cli.py", line 140, in run
    DICOMSorter(arguments).start()
  File "/Users/d/miniconda2/envs/fmri/lib/python2.7/site-packages/dicomsorter/dicomsorter.py", line 32, in start
    for _ in iterator:
  File "/Users/d/miniconda2/envs/fmri/lib/python2.7/site-packages/tqdm/_tqdm.py", line 1022, in __iter__
    for obj in iterable:
  File "/Users/d/miniconda2/envs/fmri/lib/python2.7/site-packages/multiprocess/pool.py", line 673, in next
    raise value
cPickle.PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed```

[iamdamion]

Oh, and I downloaded the standalone app for mac os and it won't open. The icon starts to show up in the dock and then it immediately closes.

[suever]

That's odd, as we explicitly use pathos to prevent that error. Can you provide the command you used to install this package so I can reproduce on my end?

[iamdamion]

Sure, I used pip to install:

pip install dicomsorter==0.1.0a3

Oh, and I just checked a different laptop it was installed on that recently got updated to mojave as well. Same issue.

Let me know if there’s any other info I can provide.

[suever]

@iamdamion Do you mind trying to install the latest version (just created based on HEAD of `master):

pip install dicomsorter==0.1.0a4

[iamdamion]

Hmm, same error. Double checked install, looks correct. Here's both:

Version Check: dicomsorter --version dicomsorter 0.1.0a4

Error: File "/Users/d/miniconda2/envs/fmri/bin/dicomsorter", line 10, in sys.exit(main()) File "/Users/d/miniconda2/envs/fmri/lib/python2.7/site-packages/dicomsorter/cli.py", line 146, in main run() File "/Users/d/miniconda2/envs/fmri/lib/python2.7/site-packages/dicomsorter/cli.py", line 140, in run DICOMSorter(**vars(arguments)).start() File "/Users/d/miniconda2/envs/fmri/lib/python2.7/site-packages/dicomsorter/dicomsorter.py", line 48, in start for _ in iterator: File "/Users/d/miniconda2/envs/fmri/lib/python2.7/site-packages/tqdm/_tqdm.py", line 1022, in iter for obj in iterable: File "/Users/d/miniconda2/envs/fmri/lib/python2.7/site-packages/multiprocess/pool.py", line 673, in next raise value cPickle.PicklingError: Can't pickle <type 'function'>: attribute lookup builtin.function failed

[suever]

@iamdamion Can you show me what version of python you're running (python --version) and also what the output of pip freeze is so I can see what versions of the dependencies you are using?

[suever]

It may be related to https://github.com/uqfoundation/pathos/issues/134

What happens when you try the following:

>>> import _multiprocess

[iamdamion]

Yeah, I'm on python 2.7. I get the same import error as in the link you sent. But my big question is why would that change? This conda env is the same one I use on all my computers so my analyses/scripts are the same. The new OS is the only real change I can think of. But this multiprocess import issue is what's going on (I think..)

import _multiprocess ImportError: No module named _multiprocess

python --version Python 2.7.15

pip freeze DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. apipkg==1.5 appnope==0.1.0 asn1crypto==0.24.0 attrs==17.4.0 backports-abc==0.5 backports.functools-lru-cache==1.5 backports.lzma==0.0.13 backports.shutil-get-terminal-size==1.0.0 backports.ssl-match-hostname==3.5.0.1 bctpy==0.5.0 bleach==1.4.2 certifi==2018.1.18 cffi==1.11.5 chardet==3.0.4 click==6.7 configparser==3.5.0 cryptography==2.1.4 cycler==0.10.0 Cython==0.29.6 dcm2bids==1.1.8 decorator==4.2.1 dicomsorter==0.1.0a4 dill==0.2.9 entrypoints==0.2.3 enum34==1.1.6 execnet==1.5.0 fastcache==1.0.2 fasteners==0.14.1 funcsigs==1.0.2 functools32==3.2.3.post2 future==0.16.0 gevent==1.3a1 gmpy2==2.0.8 greenlet==0.4.13 h5py==2.7.1 html5lib==1.0.1 idna==2.6 ipaddress==1.0.18 ipykernel==4.8.2 ipython==5.8.0 ipython-genutils==0.2.0 ipywidgets==7.1.2 isodate==0.6.0 Jinja2==2.10 jsonschema==2.6.0 jupyter==1.0.0 jupyter-client==5.2.2 jupyter-console==5.2.0 jupyter-core==4.4.0 keepalive==0.5 kiwisolver==1.0.1 LabJackPython===10-22-2012 lxml==4.1.1 MarkupSafe==1.0 matplotlib==2.2.2 mistune==0.8.3 mne==0.16.2 mock==2.0.0 monotonic==1.5 msgpack==0.5.6 multiprocess==0.70.7 nbconvert==5.3.1 nbformat==4.4.0 networkx==2.1 nibabel==2.3.0 nilearn==0.4.2 nipy==0.4.1 nipype==1.1.1 notebook==5.4.0 numexpr==2.6.4 numpy==1.14.5 olefile==0.45.1 packaging==17.1 pandas==0.23.3 pandocfilters==1.4.1 pathlib==1.0.1 pathlib2==2.3.0 pathos==0.2.3 patsy==0.5.0 pbr==3.1.1 pexpect==4.4.0 pickleshare==0.7.4 Pillow==5.0.0 plotly==3.1.0 pluggy==0.6.0 pox==0.2.5 ppft==1.6.4.9 prompt-toolkit==1.0.15 prov==1.5.0 psutil==5.4.6 PsychoPy==1.82.1 ptitprince==0.1.4 ptyprocess==0.5.2 py==1.5.2 pycparser==2.18 pydicom==1.2.0 pydotplus==2.0.2 pyglet==1.2.4 Pygments==2.2.0 PyHamcrest==1.9.0 pymvpa2==2.6.4 pyobjc-core==4.2.2 pyobjc-framework-Cocoa==4.0 pyobjc-framework-Quartz==4.2.2 pyOpenSSL==17.5.0 pyparsing==2.2.0 pyqtgraph==0.10.0 PySocks==1.6.8 pytest==3.4.1 pytest-forked==0.2 pytest-xdist==1.22.5 python-dateutil==2.7.3 pytz==2018.3 pyzmq==17.0.0 qtconsole==4.3.1 rdflib==4.2.2 requests==2.18.4 retrying==1.3.3 scandir==1.7 scikit-learn==0.19.1 scipy==1.1.0 seaborn==0.9.0 Send2Trash==1.5.0 simplegeneric==0.8.1 simplejson==3.8.1 singledispatch==3.4.0.3 six==1.11.0 SPARQLWrapper==1.8.0 statsmodels==0.8.0 subprocess32==3.2.7 sympy==1.1.1 tables==3.4.4 terminado==0.8.1 testpath==0.3.1 tornado==4.5.3 tqdm==4.31.1 traitlets==4.3.2 traits==4.6.0 urllib3==1.22 wcwidth==0.1.7 webencodings==0.5 widgetsnbextension==3.1.4 wxPython==4.0.2 xvfbwrapper==0.2.9

[suever]

So it looks like this error is caused due to their not being a C compiler found on the machine when multiprocess is being installed which causes it to not be installed correctly.

Previously, you would use xcode-select --install to install the Command Line Tools. I have them running on my Mojave machine (I installed XCode first and then the command line tools), but it appears that the minimal way has some potential issues in Mojave.

Once you install the Command Line Tools, maybe retry installing multiprocess and hopefully it will actually compile. It's frustrating that there isn't a hard error when installing multiprocess and instead is a silent and unexpected change in behavior.

[iamdamion]

Well, after a deep dive, it looks like this fixed the new computer install. However, one laptop that was upgraded to mojave still gives this error. Even after re-installing xcode, command line tools, and multiprocess.

I've seen a few posts of people using dill instead, but it looks like this is more an osx upgrade issue than anything else. I really appreciate your help with this! I'll keep messing with things on my end, but I might be stuck only using one of our machines until some upgrade or change happens on the OS side....

[suever]

Glad you were able to get it at least partly working. I'll investigate some alternatives on my end (including dill) and let you know if I can come up with a more system-independent way of doing things.

[iamdamion]

Yeah, it's very confusing. I can't find any version differences in xcode, xquartz, command line tools, and all python libraries are identical (conda env). The only difference is that one was upgraded to mojave and the other is a new computer that came with mojave installed. Thanks again!

[suever]

Going to go ahead and close this out. Feel free to open again if you continue to experience any issues!