Windows and Cygwin __pycache__ entries conflict

[pyhedgehog]

If I want to check same version of windows and cygwin flavour of python one that's tested later always fail. Removing of __pycache__ directories helps. Error is like this:

# python -m pytest tests/test_cli.py
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/_pytest/config.py", line 362, in _importconftest
    mod = conftestpath.pyimport()
  File "/usr/lib/python2.7/site-packages/py/_path/local.py", line 680, in pyimport
    raise self.ImportMismatchError(modname, modfile, self)
ImportMismatchError: ('conftest', 'C:\\src\\project\\tests\\conftest.py', local('/src/github/project/tests/conftest.py'))
ERROR: could not load /src/project/tests/conftest.py

[nicoddemus]

I'm not sure this is solvable by pytest, sorry. But thanks for taking the time to write anyway.

[pyhedgehog]

Do you mean that this should be reported to https://github.com/pytest-dev/py? Ok: https://github.com/pytest-dev/py/issues/113.

[nicoddemus]

No, I'm not sure this is solvable at all: at a glance there's nothing on PEP-3147 that would allow an implementation to distinguish between platforms. Sorry if I was too terse before.

[RonnyPfannschmidt]

@nicoddemus as far as i can tell this happens because python 2.7 has no PEP-3147

@pyhedgehog does this happen with python3 as well or is it limited to python2.7

[pyhedgehog]

@RonnyPfannschmidt no, it happens with python3 as well. That's exactly reason why I post it here - python2 itself doesn't supports PEP-3147, but py/pytest forces it to to write to __pycache__. So it can add platform name to cached name.

[RonnyPfannschmidt]

If it happens in all Python versions, its likely a Python bug, should we work around?

[nicoddemus]

I just realized a workaround, setting PYTHONDONTWRITEBYTECODE=1 will prevent pytest from writing pyc files. Might be less efficient, but I doubt it is noticeable.

[pyhedgehog]

@RonnyPfannschmidt No. In python2 only pytest writes to __pycache__ - python itself writes .pyc files beside .py.

@nicoddemus Thanks!

[pyhedgehog]

I propose change here:

- PYTEST_TAG = imp.get_tag() + "-PYTEST"
+ PYTEST_TAG = imp.get_tag() + "-" + sys.platform + "-PYTEST"
...
- PYTEST_TAG = "%s-%s%s-PYTEST" % (impl, ver[0], ver[1])
+ PYTEST_TAG = "%s-%s%s-%s-PYTEST" % (impl, ver[0], ver[1], sys.platform)

[RonnyPfannschmidt]

@pyhedgehog thanks for the hint, but i fear by doing that we would be in direct breach of the related PEP

i'd really like to get the input of @benjaminp on this one

[pyhedgehog]

@RonnyPfannschmidt Doesn't -PYTEST tail already breaks this PEP?

[jaraco]

I've encountered this issue also when I use VMWare Fusion to run Windows mounted to my projects directory. When I try to run the tests in Windows after running them in macOS, the __file__ attribute triggers the error:

PS C:\Users\jaraco\m\jaraco.windows> tox -r
python recreate: C:\Users\jaraco\m\jaraco.windows\.tox\python
python installdeps: setuptools>=31.0.1
python develop-inst: C:\Users\jaraco\m\jaraco.windows
python installed: You are using pip version 9.0.1, however version 10.0.0 is available.,You should consider upgrading via the 'python -m pip install --upgrade pip' command.,attrs==17.4.0,collective.checkdocs==0.2,colorama==0.3.9,docutils==0.14,flake8==3.5.0,jaraco.classes==1.4.3,jaraco.collections==1.5.3,jaraco.functools==1.17,jaraco.structures==1.1.2,jaraco.text==1.10,jaraco.ui==1.6,-e git+gh://jaraco/jaraco.windows@53b2bc85f580292f0361f2804eefab7f994e6874#egg=jaraco.windows,mccabe==0.6.1,more-itertools==4.1.0,path.py==11.0.1,pluggy==0.6.0,py==1.5.3,pycodestyle==2.3.1,pyflakes==1.6.0,pytest==3.5.0,pytest-flake8==1.0.0,pytest-sugar==0.9.1,six==1.11.0,termcolor==1.1.0
python runtests: PYTHONHASHSEED='447'
python runtests: commands[0] | py.test
Test session starts (platform: win32, Python 3.6.4, pytest 3.5.0, pytest-sugar 0.9.1)
rootdir: C:\Users\jaraco\m\jaraco.windows, inifile: pytest.ini
plugins: sugar-0.9.1, flake8-1.0.0

――――――――――――――――――――― ERROR collecting tests/test_root.py ―――――――――――――――――――――
.tox\python\lib\site-packages\py\_path\local.py:686: in pyimport
    raise self.ImportMismatchError(modname, modfile, self)
E   py._path.local.LocalPath.ImportMismatchError: ('test_root', '/Users/jaraco/Dropbox/code/main/jaraco.windows/tests/test_root.py', local('C:\\Users\\jaraco\\m\\jaraco.windows\\tests\\test_root.py'))

――――――――――――――――――――― ERROR collecting tests/test_root.py ―――――――――――――――――――――
import file mismatch:
imported module 'test_root' has this __file__ attribute:
  /Users/jaraco/Dropbox/code/main/jaraco.windows/tests/test_root.py
which is not the same as the test file we want to collect:
  C:\Users\jaraco\m\jaraco.windows\tests\test_root.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules

!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!

Results (14.63s):
ERROR: InvocationError: 'C:\\Users\\jaraco\\m\\jaraco.windows\\.tox\\python\\Scripts\\py.test.EXE'
______________________________________________________ summary _______________________________________________________
ERROR:   python: commands failed

I have to find . -name \*.pyc | xargs rm and then the tests can proceed. I have to do the same switching back from Windows to macOS on the project.

[jaraco]

I've also seen this issue between macOS and Linux when using Vagrant mounted to the project directory.