dill ImportError: cannot import name source

[jacopo-chevallard]

Hi, I'm running Python 2.7.15 and just updated pathos (to version 0.2.2.1) and dependencies (dill-0.2.8.2, multiprocess-0.70.6.1), and unfortunately I'm getting this strange error

Traceback (most recent call last):
  File "test.py", line 3, in <module>
    from pathos.pools import ProcessPool as Pool
  File "/Users/jchevall/anaconda/lib/python2.7/site-packages/pathos/__init__.py", line 55, in <module>
    from . import pools
  File "/Users/jchevall/anaconda/lib/python2.7/site-packages/pathos/pools.py", line 12, in <module>
    from pathos.helpers import ProcessPool as _ProcessPool
  File "/Users/jchevall/anaconda/lib/python2.7/site-packages/pathos/helpers/__init__.py", line 9, in <module>
    from . import pp_helper
  File "/Users/jchevall/anaconda/lib/python2.7/site-packages/pathos/helpers/pp_helper.py", line 33, in <module>
    import dill as pickle
  File "/Users/jchevall/anaconda/lib/python2.7/site-packages/dill/__init__.py", line 29, in <module>
    from . import source, temp, detect
ImportError: cannot import name source

when running this simple example

import random
from pathos.pools import ProcessPool as Pool

def process(item):
    return item.power()

class A(object):
    def __init__(self, num):
        super(A, self).__init__()
        self.num = num

    def power(self):
        self.num = self.num ** 2
        return self.num

pool = Pool(3)
items = [A(random.uniform(1,10)) for x in range(1,3)]

results = pool.map(process, items)
print(results)

any hint?

[jacopo-chevallard]

I could make the test work by running pip uninstall multiprocess dill pathos and reinstalling with conda install -c conda-forge pathos, I have now multiprocess-0.70.5, dill-0.2.8.2, and pathos-0.2.1

[mmckerns]

That's weird... hmm. Thanks for your report. I just tried your code with your exact package configuration, and I'm not seeing an error. So the only thing I can think of is... is there some weird naming conflict? You'll notice that the import is from . import source..., so that means it's supposed to be looking in the same directory for source.py, which is actually dill.source. That error you are seeing is coming from dill... and the version of dill in both cases is the same for you... so that's terribly strange.

[mmckerns]

The difference between the versions of pathos is primarily for python 3.7 support, so that shouldn't also be an issue. The update to multiprocess was significant, in that I updated the 2.7 support to python 2.7.15, so that might be something if it's not an $PATH or naming issue.

[jacopo-chevallard]

Thanks for the quick feedback, I'll take a look at potential path/naming issues and report back

[mmckerns]

@jacopo-chevallard: Any resolution on this on your side? Since I didn't hear back from you, can I assume it was resolved?

[mmckerns]

After rereading this thread, here's what I expect happened... I expect you have a pip install and a conda install, and they are different installations -- both in different directories. That can lead to weird import conflicts and failures, link conflicts and failures, and versioning conflicts and failures. I expect the uninstall and reinstall with conda resolved it either by fixing a broken install or resync'ing a broken dependency chain due to having two different installs seen on a single PATH.

Please feel free to reopen if the issue is not resolved.