search

alexsilva / supervisor

Supervisor process control system for Windows
http://supervisord.org
Other
118 stars 26 forks source link

running "python -m supervisor.services" raise ModuleNotFoundError errors #34

Open yahman72 opened 3 years ago

yahman72 commented 3 years ago

Environment

When running python -m supervisor.services (remove or install)

ModuleNotFoundError errors are raised, e.g.

NOTE: To me it looks like that the problems are caused by the pywin32 package (i.e. pywin32 bugs or how pywin32 is used in supervisor-win). --> this issue might end up as "WONTFOX" but I'll document my findings here anyway for the others who might have the same problem.

There are several scenarios where this happens

Scenario-1: virtualenv & supervisor-win as depency to "develop" install

Steps to reprocude: 0) create and activate a python virtualenv 1) in new python package ("my_package") define supervisor-win as dependency i.e.

setup(
  ...
  "install_requires": ["supervisor-win"],
  ...
  )

2) install that "my_package" in "develop" mode (i.e. python setup.py develop) i.e. after this pip list shows my package

> pip list
Package            Version    Location
------------------ ---------- -----------------
my_package         0.0.0.dev1 path\to\my_package\src

3) run python -m supervisor.services remove --> FAIL

> python -m supervisor.services remove
Traceback (most recent call last):
  File "C:\Python37\Lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Python37\Lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "c:\ta\virtualenv\my_venv_1\lib\site-packages\supervisor_win-4.4.0-py3.7.egg\supervisor\services.py", line 13, in <module>
    import servicemanager
ModuleNotFoundError: No module named 'servicemanager'

(possible) problem, dependencies are installed as eggs = do not work

> cd %VIRTUAL_ENV% & find . | egrep "supervisor.*services|servicemanager"
.\Lib\site-packages\pywin32-228-py3.7-win32.egg\win32\servicemanager.pyd
.\Lib\site-packages\supervisor_win-4.4.0-py3.7.egg\supervisor\services.py
.\Lib\site-packages\supervisor_win-4.4.0-py3.7.egg\supervisor\__pycache__\services.cpython-37.pyc

Workaround: Install supervisor-win and pywin32 manually i.e. run

pip uninstall supervisor_win pywin32
pip install supervisor_win==4.4.0
pip install pywin32==228

Now the command works:

> python -m supervisor.services remove
Removing service Supervisor Pyv3.7-32
Error removing service: The specified service does not exist as an installed service. (1060)

... and the difference is how the packages are installed i.e. not eggs

> cd %VIRTUAL_ENV% & find . | egrep "supervisor.*services|servicemanager"
.\Lib\site-packages\supervisor\services.py
.\Lib\site-packages\supervisor\__pycache__\services.cpython-37.pyc
.\Lib\site-packages\win32\servicemanager.pyd

For comparison: If I install that "my_package" from a wheel to anothter virtualenv then it works out of the box. i.e. pip list shows my package now like this

> pip list
Package            Version   
------------------ ----------
my_package         0.0.0.dev1

again the command works:

> python -m supervisor.services remove
Removing service Supervisor Pyv3.7-32
Error removing service: The specified service does not exist as an installed service. (1060)

... and the packages here are also not eggs

> cd %VIRTUAL_ENV% & find . | egrep "supervisor.*services|servicemanager"
.\Lib\site-packages\supervisor\services.py
.\Lib\site-packages\supervisor\__pycache__\services.cpython-37.pyc
.\Lib\site-packages\win32\servicemanager.pyd

Scenario-2: no virtualenv & supervisor-win as depency to wheel install

Steps to reprocude: 0) in global python env (no virtualenv) 1) in new python package ("my_package") define supervisor-win as dependency i.e.

setup(
  ...
  "install_requires": ["supervisor-win"],
  ...
  )

2) create a wheel for "my_package" (i.e. python setup.py bdist_wheel) 3) install that that wheel with pip (i.e. pip install path\to\my_package.whl) i.e. after this pip list shows my package

> pip list
Package            Version   
------------------ ----------
my_package         0.0.0.dev1

3) run python -m supervisor.services remove --> FAIL

> python -m supervisor.services remove
===== command execution failed ====
Traceback (most recent call last):
  File "C:\Python37\lib\site-packages\supervisor\services.py", line 303, in <module>
    main()
  File "C:\Python37\lib\site-packages\supervisor\services.py", line 298, in main
    win32serviceutil.HandleCommandLine(SupervisorService, argv=srv_argv)
  File "C:\Python37\lib\site-packages\win32\lib\win32serviceutil.py", line 551, in HandleCommandLine
    import getopt
ModuleNotFoundError: No module named 'getopt'

This time the egg-distributions cannot be the problem:

> find %PYTHONBINPATH% | egrep "supervisor.*services|servicemanager|getopt"
C:\Python37\include\internal\pygetopt.h
C:\Python37\Lib\distutils\fancy_getopt.py
C:\Python37\Lib\distutils\__pycache__\fancy_getopt.cpython-37.opt-1.pyc
C:\Python37\Lib\distutils\__pycache__\fancy_getopt.cpython-37.opt-2.pyc
C:\Python37\Lib\distutils\__pycache__\fancy_getopt.cpython-37.pyc
C:\Python37\Lib\getopt.py
C:\Python37\Lib\site-packages\supervisor\services.py
C:\Python37\Lib\site-packages\supervisor\__pycache__\services.cpython-37.pyc
C:\Python37\Lib\site-packages\win32\servicemanager.pyd
C:\Python37\Lib\test\test_getopt.py
C:\Python37\Lib\test\__pycache__\test_getopt.cpython-37.opt-1.pyc
C:\Python37\Lib\test\__pycache__\test_getopt.cpython-37.opt-2.pyc
C:\Python37\Lib\test\__pycache__\test_getopt.cpython-37.pyc
C:\Python37\Lib\__pycache__\getopt.cpython-37.opt-1.pyc
C:\Python37\Lib\__pycache__\getopt.cpython-37.opt-2.pyc
C:\Python37\Lib\__pycache__\getopt.cpython-37.pyc

Workaround: The workaround from #12 helps here also: add PYTHONBINPATH to PYTHONPATH env variable i.e. 

set PYTHONPATH=%PYTHONBINPATH%

However this is quite ugly workaround, that might cause issues elsewhere because that changes the default search order i.e. without PYTHONPATH

> python
Python 3.7.7 (tags/v3.7.7:d7c567b08f, Mar 10 2020, 09:44:33) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print("\n".join(sys.path))

C:\Python37\python37.zip
C:\Python37\DLLs
C:\Python37\lib
C:\Python37
C:\Python37\lib\site-packages
C:\Python37\lib\site-packages\win32
C:\Python37\lib\site-packages\win32\lib
C:\Python37\lib\site-packages\Pythonwin

with PYTHONPATH=%PYTHONBINPATH%

> python
Python 3.7.7 (tags/v3.7.7:d7c567b08f, Mar 10 2020, 09:44:33) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print("\n".join(sys.path))

C:\Python37
C:\Python37\python37.zip
C:\Python37\DLLs
C:\Python37\lib
C:\Python37\lib\site-packages
C:\Python37\lib\site-packages\win32
C:\Python37\lib\site-packages\win32\lib
C:\Python37\lib\site-packages\Pythonwin
alexsilva commented 3 years ago

It can be the cause of many problems. e8a7c4b

gerroon commented 3 years ago

I get ModuleNotFoundError: No module named 'locale' Is this also related to this one?

femarschall commented 3 years ago

I also have this error and I tried to solve it with following code but failed: pip install Locales

I reinstalled supervisord from github and then I was able to install it as a Windows Service python -m pip install git+https://github.com/alexsilva/supervisor@windows -U python -m supervisor.services install -c C:\python\supervisord\supervisord.conf

Still, Windows Service was crashing because I didn't replace %(ENV_TMP) from config file. After it was fixed Service was allowed to start and web interface opens