virtualenv option doesn't invoke correct Python3

[jkugler]

I'm working on getting a virtualenv package built.

• My first attempt resulted in realizing a I needed to install virtualenv-tools.
• I then realized I needed virtualenv-tools3. :)

Virtualenv-tools ran, but failed with:

Error in ./lib/python3.11/site-packages/rest_framework_simplejwt/__pycache__/compat.cpython-311.pyc {:level=>:info}
Traceback (most recent call last): {:level=>:info}
  File "/home/jkugler/.local/bin/virtualenv-tools", line 8, in <module> {:level=>:info}
    sys.exit(main()) {:level=>:info}
  File "/home/jkugler/.local/lib/python3.10/site-packages/virtualenv_tools.py", line 357, in main {:level=>:info}
    update_paths(venv, update_path) {:level=>:info}
  File "/home/jkugler/.local/lib/python3.10/site-packages/virtualenv_tools.py", line 230, in update_paths {:level=>:info}
    update_pycs(lib_dir, new_path) {:level=>:info}
  File "/home/jkugler/.local/lib/python3.10/site-packages/virtualenv_tools.py", line 180, in update_pycs {:level=>:info}
    update_pyc(filename, local_path) {:level=>:info}
  File "/home/jkugler/.local/lib/python3.10/site-packages/virtualenv_tools.py", line 122, in update_pyc {:level=>:info}
    code = marshal.load(f) {:level=>:info}
ValueError: bad marshal data (unknown type code) {:level=>:info}
Process failed: virtualenv-tools failed (exit code 1). Full command was:["virtualenv-tools", "--update-path", "/path/to/my.whl"] {:level=>:error}

Looking through the verbose output, it can be seen that it used Python 3.11 to invoke the virtualenv command because $(which virtualenv) is:

#!/usr/bin/python3.11
# -*- coding: utf-8 -*-
import re
import sys
from virtualenv.__main__ import run_with_catch
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(run_with_catch())

However, $(which virtualenv-tools) is:

#!/usr/bin/python3.10
# -*- coding: utf-8 -*-
import re
import sys
from virtualenv_tools import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

I am guessing FPM shells out to those command, hence the issue. A few suggestions for improvements/fixes.

1. Add a --virtualenv-python-binary option so the binary to be used for the virtualenv operation can be specified.
2. For creating the virtualenv, do no use virtualenv but $PYTHON_BINARY -m venv /path/to/venv
3. For invoking virtualenv-tools, it can also be done $PYTHON_BINARY -m virtualenv-tools [options go here]

That would accomplish (at least) two things.

1. Make it apparent if a dependency (e.g. virtualenv-tools isn't installed under $PYTHON_BINARY's lib path
2. All commands would be invoked by the desired $PYTHON_BINARY, compiled to pyc by the desired $PYTHON_BINARY, etc.

[jkugler]

I made sure both virtualenv and virtualenv-tools where installed by python3.11, and re-ran but then got this:

Error in ./lib/python3.11/site-packages/prometheus_client/openmetrics/__pycache__/parser.cpython-310.pyc {:level=>:info}
Traceback (most recent call last): {:level=>:info}
  File "/home/jkugler/.local/bin/virtualenv-tools", line 8, in <module> {:level=>:info}
    sys.exit(main()) {:level=>:info}
             ^^^^^^ {:level=>:info}
  File "/home/jkugler/.local/lib/python3.11/site-packages/virtualenv_tools.py", line 357, in main {:level=>:info}
    update_paths(venv, update_path) {:level=>:info}
  File "/home/jkugler/.local/lib/python3.11/site-packages/virtualenv_tools.py", line 230, in update_paths {:level=>:info}
    update_pycs(lib_dir, new_path) {:level=>:info}
  File "/home/jkugler/.local/lib/python3.11/site-packages/virtualenv_tools.py", line 180, in update_pycs {:level=>:info}
    update_pyc(filename, local_path) {:level=>:info}
  File "/home/jkugler/.local/lib/python3.11/site-packages/virtualenv_tools.py", line 122, in update_pyc {:level=>:info}
    code = marshal.load(f) {:level=>:info}
           ^^^^^^^^^^^^^^^ {:level=>:info}
ValueError: bad marshal data (unknown type code) {:level=>:info}
Process failed: virtualenv-tools failed (exit code 1). Full command was:["virtualenv-tools", "--update-path", "/path/to/my.whl"] {:level=>:error}

Something in the chain is still invoking python 3.10...and I'm not sure what. I believe the $PYTHON_BINARY -m suggestion would help with this.

[jkugler]

OK...while the above suggestions would be spiffy (as would the ability to specify the Python version of the virtualenv to create), it appears something is pulling in 3.10 pyc files even though I'm running in a 3.11 virtualenv. I was able to replicate this with a bare pip3 install in a virtualenv. I'll close this for now. Might open a new issue about adding some options to the virutalenv module. :)