FileNotFoundError when restarting test with usedevelop=True

[ashwinvis]

Traceback

(helloworld-py-lNkW4Xyt) sandbox/helloworld-py » tox -e py36
py36 create: /home/avmo/src/sandbox/helloworld-py/.tox/py36
WARNING:test command found but not installed in testenv
  cmd: /scratch/avmo/opt/helloworld-py-lNkW4Xyt/bin/python
  env: /home/avmo/src/sandbox/helloworld-py/.tox/py36
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
py36 develop-inst: /home/avmo/src/sandbox/helloworld-py
WARNING:test command found but not installed in testenv
  cmd: /scratch/avmo/opt/helloworld-py-lNkW4Xyt/bin/pip
  env: /home/avmo/src/sandbox/helloworld-py/.tox/py36
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
WARNING:test command found but not installed in testenv
  cmd: /scratch/avmo/opt/helloworld-py-lNkW4Xyt/bin/pip
  env: /home/avmo/src/sandbox/helloworld-py/.tox/py36
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
py36 installed: ----------------------------------------,Error when trying to get requirement for VCS system Command "git config --get-regexp remote\..*\.url" failed with error code 1 in /home/avmo/src/sandbox/helloworld-py, falling back to uneditable format,Could not determine repository location of /home/avmo/src/sandbox/helloworld-py,## !! Could not determine repository location,HelloWorld==0.1,pew==1.1.2,pipenv==11.0.2,pluggy==0.6.0,py==1.5.2,six==1.11.0,tox==2.9.1,tox-pipenv==1.4.0,virtualenv==15.1.0,virtualenv-clone==0.3.0
py36 runtests: PYTHONHASHSEED=None
py36 runtests: commands[0] | python test.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
______________________________________________________________________________________________ summary _______________________________________________________________________________________________
  py36: commands succeeded
  congratulations :)
(helloworld-py-lNkW4Xyt) sandbox/helloworld-py » tox -e py36
WARNING:test command found but not installed in testenv
  cmd: /scratch/avmo/opt/helloworld-py-lNkW4Xyt/bin/python
  env: /home/avmo/src/sandbox/helloworld-py/.tox/py36
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
ERROR: invocation failed (errno 2), args: [local('/home/avmo/src/sandbox/helloworld-py/.tox/py36/bin/python'), '/home/avmo/src/sandbox/helloworld-py/setup.py', '--name'], cwd: /home/avmo/src/sandbox/helloworld-py
Traceback (most recent call last):
  File "/scratch/avmo/opt/helloworld-py-lNkW4Xyt/bin/tox", line 11, in <module>
    sys.exit(cmdline())
  File "/scratch/avmo/opt/helloworld-py-lNkW4Xyt/lib/python3.6/site-packages/tox/session.py", line 40, in main
    retcode = Session(config).runcommand()
  File "/scratch/avmo/opt/helloworld-py-lNkW4Xyt/lib/python3.6/site-packages/tox/session.py", line 392, in runcommand
    return self.subcommand_test()
  File "/scratch/avmo/opt/helloworld-py-lNkW4Xyt/lib/python3.6/site-packages/tox/session.py", line 563, in subcommand_test
    self.developpkg(venv, self.config.setupdir)
  File "/scratch/avmo/opt/helloworld-py-lNkW4Xyt/lib/python3.6/site-packages/tox/session.py", line 489, in developpkg
    venv.developpkg(setupdir, action)
  File "/scratch/avmo/opt/helloworld-py-lNkW4Xyt/lib/python3.6/site-packages/tox/venv.py", line 249, in developpkg
    if not self._needs_reinstall(setupdir, action):
  File "/scratch/avmo/opt/helloworld-py-lNkW4Xyt/lib/python3.6/site-packages/tox/venv.py", line 222, in _needs_reinstall
    returnout=True, env=env)
  File "/scratch/avmo/opt/helloworld-py-lNkW4Xyt/lib/python3.6/site-packages/tox/session.py", line 140, in popen
    stdout=stdout, stderr=subprocess.STDOUT)
  File "/scratch/avmo/opt/helloworld-py-lNkW4Xyt/lib/python3.6/site-packages/tox/session.py", line 228, in _popen
    stdout=stdout, stderr=stderr, env=env)
  File "/usr/lib64/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib64/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '.tox/py36/bin/python': '.tox/py36/bin/python'

Steps to reproduce

git clone https://github.com/ashwinvis/helloworld-py.git
cd helloworld-py
pipenv install -e .
pipenv shell
tox -e py36  # Some warnings like "WARNING:test command found but not installed in testenv"
tox -e py36  # FileNotFoundError

Workarounds as of now

1. Run tests simply as tox without the -e py36 option
2. Set usedevelop=False in tox.ini

[tonybaloney]

hi @ashwinvis thanks for raising the issue.

[tonybaloney]

It looks like you're using tox from within the pipenv shell, it should be called outside. tox-pipenv calls pipenv itself.

[tonybaloney]

followed your steps and called from outside the pipenv shell.

(env) ➜  helloworld-py git:(master) ✗ python3.6
Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 26 2018, 19:50:54) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
(env) ➜  helloworld-py git:(master) ✗ tox -e py36
py36 create: /Users/anthonyshaw/repo/test-36/helloworld-py/.tox/py36
WARNING:test command found but not installed in testenv
  cmd: /Users/anthonyshaw/repo/test-36/env/bin/python
  env: /Users/anthonyshaw/repo/test-36/helloworld-py/.tox/py36
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
py36 develop-inst: /Users/anthonyshaw/repo/test-36/helloworld-py
WARNING:test command found but not installed in testenv
  cmd: /Users/anthonyshaw/repo/test-36/env/bin/pip
  env: /Users/anthonyshaw/repo/test-36/helloworld-py/.tox/py36
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
py36 installed: actionid: py36,msg: envreport,cmdargs: ['/Users/anthonyshaw/repo/test-36/env/bin/python3.7', '-m', 'pipenv', 'graph'],,,
py36 runtests: PYTHONHASHSEED=None
py36 runtests: commands[0] | python test.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
_____________________________________________________________________________________ summary _____________________________________________________________________________________
  py36: commands succeeded
  congratulations :)

[ashwinvis]

Noooo. Run the tox command twice, then you might see the error.

Sent from ProtonMail mobile

-------- Original Message -------- On 4 Jul 2018 1:49 am, Anthony Shaw wrote:

followed your steps and called from outside the pipenv shell.

(env) ➜ helloworld-py git:(master) ✗ python3.6 Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 26 2018, 19:50:54) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information.

exit() (env) ➜ helloworld-py git:(master) ✗ tox -e py36 py36 create: /Users/anthonyshaw/repo/test-36/helloworld-py/.tox/py36 WARNING:test command found but not installed in testenv cmd: /Users/anthonyshaw/repo/test-36/env/bin/python env: /Users/anthonyshaw/repo/test-36/helloworld-py/.tox/py36 Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting. py36 develop-inst: /Users/anthonyshaw/repo/test-36/helloworld-py WARNING:test command found but not installed in testenv cmd: /Users/anthonyshaw/repo/test-36/env/bin/pip env: /Users/anthonyshaw/repo/test-36/helloworld-py/.tox/py36 Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting. py36 installed: actionid: py36,msg: envreport,cmdargs: ['/Users/anthonyshaw/repo/test-36/env/bin/python3.7', '-m', 'pipenv', 'graph'],,, py36 runtests: PYTHONHASHSEED=None py36 runtests: commands[0] | python test.py .

Ran 1 test in 0.000s

OK summary py36: commands succeeded congratulations :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[ashwinvis]

It looks like you're using tox from within the pipenv shell, it should be called outside. tox-pipenv calls pipenv itself.

That seems impractical. How would one install tox and tox-pipenv without a virtualenv? In the user directory?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[tonybaloney]

this plugin uses pipenv to create the virtual environments for multiple target versions of Python.

You can install Tox and Tox-Pipenv into a user directory, system or into a virtualenv.

If you run it from within pipenv, then pipenv would call tox, which then would call pipenv and it'll end up getting really confused!

[ashwinvis]

Hmmm let's try it your way then:

~/s/s/helloworld-py ❯❯❯ virtualenv testenv                                                                                                                     ✘ 130 
~/s/s/helloworld-py ❯❯❯ source testenv/bin/activate
~/s/s/helloworld-py ❯❯❯ pip install tox tox-pipenv                                                                                                  (testenv)  ✘ 130 
Collecting tox
  Using cached https://files.pythonhosted.org/packages/e6/41/4dcfd713282bf3213b0384320fa8841e4db032ddcb80bc08a540159d42a8/tox-3.0.0-py2.py3-none-any.whl
Collecting tox-pipenv
  Downloading https://files.pythonhosted.org/packages/e4/15/ec42b173574e97ebbe1288f2fb4afcc3e011d90602195e9eb152fa58fabf/tox_pipenv-1.5.0-py3-none-any.whl
Collecting py>=1.4.17 (from tox)
  Downloading https://files.pythonhosted.org/packages/f3/bd/83369ff2dee18f22f27d16b78dd651e8939825af5f8b0b83c38729069962/py-1.5.4-py2.py3-none-any.whl (83kB)
    100% |████████████████████████████████| 92kB 6.8MB/s 
Collecting virtualenv>=1.11.2 (from tox)
  Using cached https://files.pythonhosted.org/packages/b6/30/96a02b2287098b23b875bc8c2f58071c35d2efe84f747b64d523721dc2b5/virtualenv-16.0.0-py2.py3-none-any.whl
Collecting six (from tox)
  Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting pluggy<1.0,>=0.3.0 (from tox)
  Using cached https://files.pythonhosted.org/packages/ba/65/ded3bc40bbf8d887f262f150fbe1ae6637765b5c9534bd55690ed2c0b0f7/pluggy-0.6.0-py3-none-any.whl
Collecting pipenv>=2018.05.18 (from tox-pipenv)
  Downloading https://files.pythonhosted.org/packages/eb/64/9b2747d54f2008ac3dfe86c0b1c8ec126042726fd8a540d5208d26732701/pipenv-2018.7.1-py3-none-any.whl (5.0MB)
    100% |████████████████████████████████| 5.0MB 4.1MB/s 
Collecting certifi (from pipenv>=2018.05.18->tox-pipenv)
  Using cached https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl
Requirement already satisfied: pip>=9.0.1 in ./testenv/lib/python3.6/site-packages (from pipenv>=2018.05.18->tox-pipenv) (10.0.1)
Collecting virtualenv-clone>=0.2.5 (from pipenv>=2018.05.18->tox-pipenv)
  Using cached https://files.pythonhosted.org/packages/6d/c2/dccb5ccf599e0c5d1eea6acbd058af7a71384f9740179db67a9182a24798/virtualenv_clone-0.3.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=36.2.1 in ./testenv/lib/python3.6/site-packages (from pipenv>=2018.05.18->tox-pipenv) (39.2.0)
Installing collected packages: py, virtualenv, six, pluggy, tox, certifi, virtualenv-clone, pipenv, tox-pipenv
Successfully installed certifi-2018.4.16 pipenv-2018.7.1 pluggy-0.6.0 py-1.5.4 six-1.11.0 tox-3.0.0 tox-pipenv-1.5.0 virtualenv-16.0.0 virtualenv-clone-0.3.0
~/s/s/helloworld-py ❯❯❯                                                                                                                                    (testenv) 
~/s/s/helloworld-py ❯❯❯ ls                                                                                                                                 (testenv) 
HelloWorld/  HelloWorld.egg-info/  testenv/  LICENSE  Pipfile  Pipfile.lock  README.md  say_hello.py  setup.py  test.py  tox.ini
~/s/s/helloworld-py ❯❯❯ rm -rf .tox                                                                                                                   (testenv)  ✘ 2 
~/s/s/helloworld-py ❯❯❯ tox -e py36                                                                                                                        (testenv) 
py36 create: /home/avmo/src/sandbox/helloworld-py/.tox/py36
WARNING:test command found but not installed in testenv
  cmd: /home/avmo/src/sandbox/helloworld-py/testenv/bin/python
  env: /home/avmo/src/sandbox/helloworld-py/.tox/py36
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
py36 develop-inst: /home/avmo/src/sandbox/helloworld-py
WARNING:test command found but not installed in testenv
  cmd: /home/avmo/src/sandbox/helloworld-py/testenv/bin/pip
  env: /home/avmo/src/sandbox/helloworld-py/.tox/py36
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
py36 installed: actionid: py36,msg: envreport,cmdargs: ['/home/avmo/src/sandbox/helloworld-py/testenv/bin/python', '-m', 'pipenv', 'graph'],,,
py36 runtests: PYTHONHASHSEED=None
py36 runtests: commands[0] | python test.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
______________________________________________________________________________ summary _______________________________________________________________________________
  py36: commands succeeded
  congratulations :)
~/s/s/helloworld-py ❯❯❯ tox -e py36                                                                                                                        (testenv) 
WARNING:test command found but not installed in testenv
  cmd: /home/avmo/src/sandbox/helloworld-py/testenv/bin/python
  env: /home/avmo/src/sandbox/helloworld-py/.tox/py36
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
ERROR: invocation failed (errno 2), args: [local('/home/avmo/src/sandbox/helloworld-py/.tox/py36/bin/python'), '/home/avmo/src/sandbox/helloworld-py/setup.py', '--name'], cwd: /home/avmo/src/sandbox/helloworld-py
Traceback (most recent call last):
  File "/home/avmo/src/sandbox/helloworld-py/testenv/bin/tox", line 11, in <module>
    sys.exit(run_main())
  File "/home/avmo/src/sandbox/helloworld-py/testenv/lib/python3.6/site-packages/tox/session.py", line 40, in run_main
    main(args)
  File "/home/avmo/src/sandbox/helloworld-py/testenv/lib/python3.6/site-packages/tox/session.py", line 46, in main
    retcode = Session(config).runcommand()
  File "/home/avmo/src/sandbox/helloworld-py/testenv/lib/python3.6/site-packages/tox/session.py", line 415, in runcommand
    return self.subcommand_test()
  File "/home/avmo/src/sandbox/helloworld-py/testenv/lib/python3.6/site-packages/tox/session.py", line 604, in subcommand_test
    self.developpkg(venv, self.config.setupdir)
  File "/home/avmo/src/sandbox/helloworld-py/testenv/lib/python3.6/site-packages/tox/session.py", line 530, in developpkg
    venv.developpkg(setupdir, action)
  File "/home/avmo/src/sandbox/helloworld-py/testenv/lib/python3.6/site-packages/tox/venv.py", line 244, in developpkg
    if not self._needs_reinstall(setupdir, action):
  File "/home/avmo/src/sandbox/helloworld-py/testenv/lib/python3.6/site-packages/tox/venv.py", line 217, in _needs_reinstall
    returnout=True, env=env)
  File "/home/avmo/src/sandbox/helloworld-py/testenv/lib/python3.6/site-packages/tox/session.py", line 150, in popen
    stdout=stdout, stderr=subprocess.STDOUT)
  File "/home/avmo/src/sandbox/helloworld-py/testenv/lib/python3.6/site-packages/tox/session.py", line 243, in _popen
    stdout=stdout, stderr=stderr, env=env)
  File "/usr/lib64/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib64/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '.tox/py36/bin/python': '.tox/py36/bin/python'

[ashwinvis]

The reason is pipenv creates a virtualenv in .venv which is maybe not the desired behaviour for tox.

~/s/s/helloworld-py ❯❯❯ ls -la                                                                                                                        (testenv)  ✘ 1 
total 72
drwxr-xr-x 1 avmo users   256 04.07.2018 07:08 ./
drwxr-xr-x 1 avmo users  2336 23.06.2018 22:25 ../
drwxr-xr-x 1 avmo users   144 13.03.2018 11:46 .git/
drwxr-xr-x 1 avmo users    70 13.03.2018 11:37 HelloWorld/
drwxr-xr-x 1 avmo users   104 13.03.2018 11:18 HelloWorld.egg-info/
drwxr-xr-x 1 avmo users    62 04.07.2018 07:07 testenv/
drwxr-xr-x 1 avmo users    14 04.07.2018 07:08 .tox/
drwxr-xr-x 1 avmo users    72 04.07.2018 07:08 .venv/
-rw-r--r-- 1 avmo users    33 13.03.2018 11:41 .gitignore
-rw-r--r-- 1 avmo users 35147 13.03.2018 11:22 LICENSE
-rw-r--r-- 1 avmo users   221 04.07.2018 07:08 Pipfile
-rw-r--r-- 1 avmo users  5275 13.03.2018 11:26 Pipfile.lock
-rw-r--r-- 1 avmo users   110 13.03.2018 11:40 README.md
-rw-r--r-- 1 avmo users    15 13.03.2018 11:10 say_hello.py
-rw-r--r-- 1 avmo users   153 13.03.2018 11:16 setup.py
-rw-r--r-- 1 avmo users   208 13.03.2018 11:41 test.py
-rw-r--r-- 1 avmo users   116 13.03.2018 11:37 tox.ini
~/s/s/helloworld-py ❯❯❯ ls .tox/py36                                                                                                                       (testenv) 
log/
~/s/s/helloworld-py ❯❯❯ ls .venv                                                                                                                           (testenv) 
bin/  include/  lib/  lib64/  pip-selfcheck.json

[tonybaloney]

ok I see it now.

[tonybaloney]

Isolated to having usedevelop = True in tox.ini and executing twice.

[tonybaloney]

Fixed in #45

apologies for closing this early. @ashwinvis

[tonybaloney]

fixed in 1.6.0