Closed mcfoi closed 3 years ago
GrahamDumpleton
commented
4 years ago If you use:
pip install https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zipdoes module-config also generate a LoadFile directive?
If yes, what is it and does using that version make a difference?
mcfoi
commented
4 years ago Icreated a brand new virtual env in e:\venvs\devwsgi\ then activated it. Then run: pip install https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip ...output is:
Collecting https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip Using cached https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip Building wheels for collected packages: mod-wsgi Building wheel for mod-wsgi (setup.py) ... done Created wheel for mod-wsgi: filename=mod_wsgi-4.7.1-cp37-cp37m-win32.whl size=362075 sha256=d59afcff2a3cc1833436f2b9121fddc3c50b859143fd388a0784b2a7683adf11 Stored in directory: C:\Users\ADMINI~1\AppData\Local\Temp\2\pip-ephem-wheel-cache-bemifz4q\wheels\20\97\ea\37e24525d8670de23f8e1444e46fe967af81668afbc79bd7fc Successfully built mod-wsgi Installing collected packages: mod-wsgi Successfully installed mod-wsgi-4.7.1
..then I run: mod_wsgi-express module-config ...output is:
LoadFile "c:/program files (x86)/python37-32/python37.dll" LoadModule wsgi_module "e:/venvs/devwsgi/lib/site-packages/mod_wsgi/server/mod_wsgi.cp37-win32.pyd" WSGIPythonHome "e:/venvs/devwsgi"
GrahamDumpleton
commented
4 years ago But does it still fail with the same error message and encodings when used?
mcfoi
commented
4 years ago Yes
AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using 192.168.2.3. Set the 'ServerName' directive globally to suppress this message [core:warn] [pid 2192:tid 444] AH00098: pid file C:/Apache24/logs/httpd.pid overwritten -- Unclean shutdown of previous Apache run? [mpm_winnt:notice] [pid 2192:tid 444] AH00455: Apache/2.4.41 (Win32) mod_wsgi/4.7.1 Python/3.7 configured -- resuming normal operations [mpm_winnt:notice] [pid 2192:tid 444] AH00456: Apache Lounge VC14 Server built: Aug 12 2019 10:30:52 [core:notice] [pid 2192:tid 444] AH00094: Command line: 'c:\Apache24\bin\httpd.exe -d C:/Apache24' [mpm_winnt:notice] [pid 2192:tid 444] AH00418: Parent: Created child process 4472 AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using 192.168.2.3. Set the 'ServerName' directive globally to suppress this message AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using 192.168.2.3. Set the 'ServerName' directive globally to suppress this message [wsgi:info] [pid 4472:tid 432] mod_wsgi (pid=4472): Python home E:\venvs\devwsgi. [wsgi:info] [pid 4472:tid 432] mod_wsgi (pid=4472): Initializing Python. Fatal Python error: initfsencoding: unable to load the file system codec ModuleNotFoundError: No module named 'encodings'
Current thread 0x000010a4 (most recent call first): [Sun Mar 01 01:56:44.340957 2020] [mpm_winnt:crit] [pid 2192:tid 444] AH00419: master_main: create child process failed. Exiting. [Sun Mar 01 01:56:44.341957 2020] [core:info] [pid 2192:tid 444] AH00096: removed PID file C:/Apache24/logs/httpd.pid (pid=2192)
GrahamDumpleton
commented
4 years ago Are you really using 32 bit Python installation?
These days it is very hard to find a 32 bit Apache distribution and both Python and Apache need to both be 64 bit or both 32 bit.
Usually if there is a mismatch it doesn't even build, but thought I would check you really are wanting to use 32 bit and not recommended 64 bit.
mcfoi
commented
4 years ago I did a fresh installation of both Python 3.7 32bits and Apache 2.4.41 32bits as I learnt that many times, whenever native libs come in need, for windows there is NEVER (almost never) the 64bit version but just the 32bit (GDAL libs). So I took care to use 32 bit.. ..so they are double checked to be 32bits. Should I try 64bits of both?
GrahamDumpleton
commented
4 years ago How are you creating the virtual environment? If you aren't using virtualenv and are instead using python -m venv, can you try virtualenv instead. The built in way of creating virtual environments has always been a problem and has needed a fiddle to get it working in embedded systems on UNIX systems at least. The fiddle doesn't seem to be getting applied on Windows, so not sure if that is the reason.
GrahamDumpleton
commented
4 years ago Does a file e:/venvs/devwsgi/pyvenv.cfg exist? That will confirm is using python -m venv.
mcfoi
commented
4 years ago After installing Python 3.7, I installed Viertualenv with:
pip install virtualenv
then created a folder named E:\venvs and cd therin.
From E:\venvs\ I just run:
virtualenv devwsgi
to get the virtualenv created in E:\venvs\devwsgi
which I activate with
c:\>e:\venv\devwsgi\Scripts\activate.bat
mcfoi
commented
4 years ago For sake of completeness I repeated the same steps in 64bit flavour using:
Apache 2.4.41 VC14 64bit
Python 3.7.6 64bit
using virtualenv
then added the output of
mod_wsgi-express module-config
to httpd.conf
and got same error:
Fatal Python error: initfsencoding: unable to load the file system codec ModuleNotFoundError: No module named 'encodings'
mcfoi
commented
4 years ago Does a file
e:/venvs/devwsgi/pyvenv.cfgexist? That will confirm is usingpython -m venv.
I can assure I am using "virtualenv myenvname>" to create the Virtualenv.. ..but the pyvenv.cfg actually is there and its content is [for a virtualenv created with the 32bit stack (Apache & Python)]:
home = c:\program files (x86)\python37-32 implementation = CPython version_info = 3.7.6.final.0 virtualenv = 20.0.7 include-system-site-packages = false base-prefix = c:\program files (x86)\python37-32 base-exec-prefix = c:\program files (x86)\python37-32 base-executable = c:\program files (x86)\python37-32\python.exe
GrahamDumpleton
commented
4 years ago Given this isn't the first time reported recently, beginning to wonder whether the last major virtualenv version (20.x) has changed something in the structure of what it creates on Windows, such it no longer works when embedding Python in other applications.
The algorithm by which Python used to work out the location of the Python installation is really complicated and that appears to be failing to do its job.
Now although removing WSGIPythonHome may avoid the problem, it means you will need to associate your application with the virtual environment yourself in the WSGI script file using the activate_this method described in:
That presumes virtualenv still provides that activation script.
If you don't do some manually to associate it with the virtual environment, it will likely not find any extra Python modules you install in the Python virtual environment.
I'll have a think about it and perhaps enable the workaround used on UNIX systems for python -m venv type virtual environments. The issue may be that virtualenv is now trying to be compatible with how the bundled method works, but which breaks embedded systems. The issue in CPython was reported a long time ago, but has never been fixed that I remember.
mcfoi
commented
4 years ago I can confirm part of wha you say:
it will likely not find any extra Python modules you install in the Python virtual environment
Removing WSGIPythonHome allowed me to run an "Hello World" wsgi app with success.. ..but as soon as I installed Django and set up teh VirtualHost to point to its wsgy.py, I started to get errors from 'django' module not found.
So.. ..is ther any hack I can do to make it work with Django?
GrahamDumpleton
commented
4 years ago Is there an activate_this.py script file in the bin/scripts directory of the virtual environment?
GrahamDumpleton
commented
4 years ago If not, see the site.addsitedir() method in that same document I linked on virtual environments.
mcfoi
commented
4 years ago Yes The activate_this.py is in virtualenvroot/Scripts
Its content is:
# -*- coding: utf-8 -*-
"""Activate virtualenv for current interpreter:
Use exec(open(this_file).read(), {'__file__': this_file}).
This can be used when you must use an existing Python interpreter, not the virtualenv bin/python.
"""
import os
import site
import sys
try:
abs_file = os.path.abspath(__file__)
except NameError:
raise AssertionError("You must use exec(open(this_file).read(), {'__file__': this_file}))")
bin_dir = os.path.dirname(abs_file)
base = bin_dir[: -len("Scripts") - 1] # strip away the bin part from the __file__, plus the path separator
# prepend bin to PATH (this file is inside the bin directory)
os.environ["PATH"] = os.pathsep.join([bin_dir] + os.environ.get("PATH", "").split(os.pathsep))
os.environ["VIRTUAL_ENV"] = base # virtual env is right above bin directory
# add the virtual environments libraries to the host python import mechanism
prev_length = len(sys.path)
for lib in "..\Lib\site-packages".split(os.pathsep):
path = os.path.realpath(os.path.join(bin_dir, lib))
site.addsitedir(path.decode("utf-8") if "" else path)
sys.path[:] = sys.path[prev_length:] + sys.path[0:prev_length]
sys.real_prefix = sys.prefix
sys.prefix = baseSo try following instructions in:
and add to the start of the WSGI script file (project/wsgi.py file in Django):
python_home = '/usr/local/envs/myapp1'
activate_this = python_home + '/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this))Adjust the paths for the activate_this.py file as necessary.
mcfoi
commented
4 years ago python_home = '/usr/local/envs/myapp1'
What python should point this 'python_home ' var? The root of the virtualenv where I installed mod_wsgi?
GrahamDumpleton
commented
4 years ago The root of the Python virtual environment usually. That is:
e:/venvs/devwsgiAs long as activate_this ends up being the path to the activate_this.py file.
mcfoi
commented
4 years ago execfile(activate_this, dict(file=activate_this))
This should work just in Python 2.7 but not anymore in Python 3.x
For Python 3.x should be
exec(open(activate_this).read(), {'__file__': activate_this})
Right?
GrahamDumpleton
commented
4 years ago Something like that. I can't remember off hand what it should be, but yes you are right, it needs to change for Python 3.
mcfoi
commented
4 years ago Adding in Django wsgi.py in Windows
python_home = 'e:\venv\devwsgi'
activate_this = python_home + '/Scritps/activate_this.py'
exec(open(activate_this).read(), {'__file__': activate_this})activate_this.py is executed and a new env var is created: VIRTUAL_ENV pointing to root of virtualenv: e:\venv\devwsgi
Despite this, subsequent line of wsgy.py
from django.core.wsgi import get_wsgi_application
causes the error
ModuleNotFoundError: No module named 'django'
..so we still do no have Django on PYTHONPATH
GrahamDumpleton
commented
4 years ago If that is exactly what you have, you should get an exception.
Don't use backslashes in the path. Scripts is also spelt wrongly.
python_home = 'e:/venv/devwsgi'
activate_this = python_home + '/Scripts/activate_this.py'
exec(open(activate_this).read(), {'__file__': activate_this})Ensure this is at the start of the wsgi.py file before any imports.
May need to restart Apache as well to ensure fresh file is picked up.
If still no joy, add after that:
import sys
print('PATH', sys.path)What does the logs show for PATH? Does it show the site-packages directory that Django was installed into?
mcfoi
commented
4 years ago I did many tests also using suggestions from web https://stackoverflow.com/a/22477904 about adding to wsgy.py
import os, sys
# add the django project path into the sys.path
sys.path.append('E:/wsgi/meteoapi/meteoapi/meteoapi')
# add the virtualenv site-packages path to the sys.path
sys.path.append('E:/venvs/meteo64/Lib/site-packages')(Note i created a new virtualenv after upgrading to 64bit Apache/Python/mod_wsgi)
I also added the activate_this.py at top of wsgy.py
python_home = 'E:/venvs/meteo64'
activate_this = python_home + '/Scripts/activate_this.py'
exec(open(activate_this).read(), {'__file__': activate_this})By putting in wsgy.py
import sys
print('PATH', sys.path)I got the following PATH but still the ModuleNotFoundError: No module named 'django'error
PATH ['E:\\\\venvs\\\\meteo64\\\\Lib\\\\site-packages', 'E:\\\\venvs\\\\meteo64\\\\Scripts\\\\python37.zip', 'C:\\\\Program Files\\\\Python37\\\\Lib', 'C:\\\\Program Files\\\\Python37\\\\DLLs', 'C:\\\\Apache24\\\\bin', 'C:\\\\Program Files\\\\Python37', 'C:\\\\Program Files\\\\Python37\\\\lib\\\\site-packages', 'E:/wsgi/meteoapi/meteoapi/meteoapi', 'E:/venvs/meteo64/Lib/site-packages']\r
(yes: with 4 - frour - slashes)
GrahamDumpleton
commented
4 years ago You can't use sys.path.append() to add site-packages as it doesn't do all the required setup. Use site.addsitedir() as explained in:
as alternative. Specifically, something like:
python_home = ''E:/venvs/meteo64''
import sys
import site
# Calculate path to site-packages directory.
python_version = '.'.join(map(str, sys.version_info[:2]))
site_packages = python_home + '/lib/python%s/site-packages' % python_version
# Add the site-packages directory.
site.addsitedir(site_packages)There is an even more elaborate recipe described if that has issues in certain cases.
mcfoi
commented
4 years ago I could finally made at start the Django application in Windows and with Virtualenv activated.
The final Django wsgy.py
import os, sys
import site
abs_file = os.path.abspath(__file__)
base_app = os.path.dirname(os.path.dirname(abs_file))
# Virtualenv 20.x automatically adds to System Environmental variables a var named VIRTUAL_ENV
# Such var points to the curren active virtual environment, so we can get it to force loading osite-packages
VIRTUAL_ENV = os.getenv("VIRTUAL_ENV")
# add the django project path itself into the sys.path (not the 'django' module)
site.addsitedir(base_app)
# add site-packages from virtualenv, including 'django' and any other requirement
# note that just packages with their own folder set in site-packages are detected. Not simple .py scripts in /site-packages root dir.
site.addsitedir(VIRTUAL_ENV + '/Lib/site-packages')
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'meteoapi.settings')
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()This was paired with loading the mod_wsgi module in httpd.conf:
LoadModule wsgi_module "e:/venvs/meteo64/lib/site-packages/mod_wsgi/server/mod_wsgi.cp37-win_amd64.pyd"
andthe proper VirtualHost in httpd-vhosts.conf
<VirtualHost *:80>
ServerAdmin my@mail.com
ServerName myservername
ServerAlias myservername
DocumentRoot "E:/wsgi/meteoapi/"
Alias /static "E:/wsgi/meteoapi/static"
<Directory E:/wsgi/meteoapi/static>
Require all granted
</Directory>
WSGIScriptAlias / "E:/wsgi/meteoapi/meteoapi/wsgi.py"
<Directory E:/wsgi/meteoapi/meteoapi/>
AllowOverride none
Require all granted
<Files wsgi.py>
Require all granted
</Files>
</Directory>
</VirtualHost>I need someone to attempt to use:
pip install https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zipand see if this resolves the original issue with WSGIPythonHome not working.
mcfoi
commented
4 years ago On my Windows 10 + Apache 2.4 64bit+ venv Python 3.7 64bit, I first tested documented workflow:
C:\Users\mcfoi\Desktop\pip λ pip install mod_wsgi Collecting mod_wsgi Using cached mod_wsgi-4.7.1.tar.gz (498 kB) Building wheels for collected packages: mod-wsgi Building wheel for mod-wsgi (setup.py) ... done Created wheel for mod-wsgi: filename=mod_wsgi-4.7.1-cp37-cp37m-win_amd64.whl size=371415 sha256=9875f4c987bdda680b96b4bd03fe8678f9dabf6e29735e884a76c00a52dbe4f5 Stored in directory: c:\users\mcfoi\appdata\local\pip\cache\wheels\c3\b2\93\f173e3207d97ada7d56f5516cf1cdff34b38111e41d4a0bad0 Successfully built mod-wsgi Installing collected packages: mod-wsgi Successfully installed mod-wsgi-4.7.1
Running " mod_wsgi-express module-config " resulted in: λ mod_wsgi-express module-config LoadFile "c:/python374-64/python37.dll" LoadModule wsgi_module "c:/users/mcfoi/desktop/pip/venv/lib/site-packages/mod_wsgi/server/mod_wsgi.cp37-win_amd64.pyd" WSGIPythonHome "c:/users/mcfoi/desktop/pip/venv"
I did add such rows in httpd.conf and configured a Vhost <VirtualHost *:80> ServerAdmin info@cassandratech.it ServerName myapp.local ServerAlias myapp.local DocumentRoot "C:/Apache24/myapp/" Alias /static "C:/Apache24/myapp/static" <Directory C:/Apache24/myapp/static> Require all granted WSGIScriptAlias / "C:/Apache24/myapp/myapp.wsgi" <Directory C:/Apache24/myapp/> AllowOverride none Require all granted
</Directory>Having myapp.wsgi content :
def application(environ, start_response): status = '200 OK' output = b'Hello World in WSGI!' response_headers = [('Content-type', 'text/plain'), ('Content-Length', str(len(output)))] start_response(status, response_headers) return [output]
So the regular workflow seems to work in Windows 10 wher it fails in Windows 2012 R2 Server
On Mon, 9 Mar 2020 at 00:32, Graham Dumpleton notifications@github.com wrote:
I need someone to attempt to use:
pip install https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip
and see if this resolves the original issue with WSGIPythonHome not working.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/GrahamDumpleton/mod_wsgi/issues/535?email_source=notifications&email_token=AANEOSSOY722UJN65YJGZOTRGQTJFA5CNFSM4K672ADKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOFEZMA#issuecomment-596266160, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANEOSXZ6FJIBF6NCBCGM5TRGQTJFANCNFSM4K672ADA .
mcfoi
commented
4 years ago On my Windows 10 + Apache 2.4 64bit+ venv Python 3.7 64bit, I also tested the new workflow where the documented was successful:
pip install https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip
but it FAILS with a lot of output:
ERROR: Failed building wheel for mod-wsgi Running setup.py clean for mod-wsgi Failed to build mod-wsgi
On Mon, 9 Mar 2020 at 00:32, Graham Dumpleton notifications@github.com wrote:
I need someone to attempt to use:
pip install https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip
and see if this resolves the original issue with WSGIPythonHome not working.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/GrahamDumpleton/mod_wsgi/issues/535?email_source=notifications&email_token=AANEOSSOY722UJN65YJGZOTRGQTJFA5CNFSM4K672ADKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOFEZMA#issuecomment-596266160, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANEOSXZ6FJIBF6NCBCGM5TRGQTJFANCNFSM4K672ADA .
GrahamDumpleton
commented
4 years ago I don't have Windows to test, so fully expected it may fail to compile. I'll see if I can spot what I missed in the changes.
GrahamDumpleton
commented
4 years ago Try again. To be safe ensure that the local pip cache is ignored.
pip install --no-cache-dir -U https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip
shippo16
commented
4 years ago For what it's worth, here are my observations on this matter:
My system python:
>>> import sys
>>> sys.executable
'C:\\Program Files\\Python_3.8.2\\python.exe'
>>> sys.prefix
'C:\\Program Files\\Python_3.8.2'
>>> sys.base_prefix
'C:\\Program Files\\Python_3.8.2'
>>> sys.exec_prefix
'C:\\Program Files\\Python_3.8.2'
>>> sys.base_exec_prefix
'C:\\Program Files\\Python_3.8.2'
When we create a virtual environment (I'm using virtualenv),
this is what the virtual environment's interpreter holds:
>>> import sys
>>> sys.executable
'D:\\PROJ\\PYTHON\\VirtEnvs\\django\\Scripts\\python.exe'
>>> sys.prefix
'D:\\PROJ\\PYTHON\\VirtEnvs\\django'
>>> sys.base_prefix
'C:\\Program Files\\Python_3.8.2'
>>> sys.exec_prefix
'D:\\PROJ\\PYTHON\\VirtEnvs\\django'
>>> sys.base_exec_prefix
'C:\\Program Files\\Python_3.8.2'
In the virtual environment directory (on Windows) we've got "Lib" and "Scripts" directories only. Inside the "Lib" dir, there's only the "site-packages" directory, the Standard Library is not there. Clearly this is how the virtual environment's interpreter gets its Stdlib: via the two "base" prefixes.
Virtual environment interpreter:
>>> import encodings
>>> import ctypes
>>> import asyncio
>>> import collections
. . .
All good.
This is what I've added to Apache's "httpd.conf" (via "mod_wsgi-express"):
LoadModule wsgi_module "d:/proj/python/virtenvs/django/lib/site-packages/mod_wsgi/server/mod_wsgi.cp38-win_amd64.pyd"
WSGIPythonHome "d:/proj/python/virtenvs/django"
And now, this is what I get in the Apache "error.log":
Python path configuration:
PYTHONHOME = 'd:\proj\python\virtenvs\django'
. . .
sys.base_prefix = 'd:\\proj\\python\\virtenvs\\django'
sys.base_exec_prefix = 'd:\\proj\\python\\virtenvs\\django'
. . .
sys.prefix = 'd:\\proj\\python\\virtenvs\\django'
sys.exec_prefix = 'd:\\proj\\python\\virtenvs\\django'
sys.path = [
'D:\\PROJ\\PYTHON\\VirtEnvs\\django\\Scripts\\python38.zip',
'd:\\proj\\python\\virtenvs\\django\\DLLs',
'd:\\proj\\python\\virtenvs\\django\\lib',
'C:\\Apache24\\bin',
]
The interpreter that Apache is starting (the virtual environment interpreter), holds the virtual
environment home directory as both "base" prefixes. Aparently that's what is preventing Python
from "seeing" the Stdlib. And I've got the same line in "error.log":
ModuleNotFoundError: No module named 'encodings'
Now, I can figure many ways to settle this. But I'd like to get to the bottom.
Any idea of when does the virtual environment interpreter get prefixes changed and why?
Thank you! :)
shippo16
commented
4 years ago Just tried:
pip install -U https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip
Lines added to httpd.conf as per "mod_wsgi-express module-config":
LoadFile "C:/Program Files/Python_3.8.2/python38.dll"
LoadModule wsgi_module "d:/proj/python/virtenvs/graham/lib/site-packages/mod_wsgi/server/mod_wsgi.cp38-win_amd64.pyd"
WSGIPythonHome "d:/proj/python/virtenvs/graham"
Apache error.log:
[Fri Jul 17 10:52:04.425667 2020] [mpm_winnt:notice] [pid 12904:tid 588] AH00455: Apache/2.4.43 (Win64) mod_wsgi/4.7.1 Python/3.8 configured -- resuming normal operations
[Fri Jul 17 10:52:04.425667 2020] [mpm_winnt:notice] [pid 12904:tid 588] AH00456: Apache Lounge VS16 Server built: Apr 21 2020 16:23:13
[Fri Jul 17 10:52:04.425667 2020] [core:notice] [pid 12904:tid 588] AH00094: Command line: 'C:\\Apache24\\bin\\httpd.exe -d C:/Apache24'
[Fri Jul 17 10:52:04.425667 2020] [mpm_winnt:notice] [pid 12904:tid 588] AH00418: Parent: Created child process 1516
Python path configuration:
PYTHONHOME = 'd:\proj\python\virtenvs\graham'
PYTHONPATH = (not set)
program name = 'python'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = 'C:\\Apache24\\bin\\httpd.exe'
sys.base_prefix = 'd:\\proj\\python\\virtenvs\\graham'
sys.base_exec_prefix = 'd:\\proj\\python\\virtenvs\\graham'
sys.executable = 'C:\\Apache24\\bin\\httpd.exe'
sys.prefix = 'd:\\proj\\python\\virtenvs\\graham'
sys.exec_prefix = 'd:\\proj\\python\\virtenvs\\graham'
sys.path = [
'C:\\Program Files\\Python_3.8.2\\python38.zip',
'd:\\proj\\python\\virtenvs\\graham\\DLLs',
'd:\\proj\\python\\virtenvs\\graham\\lib',
'C:\\Apache24\\bin',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00000b1c (most recent call first):
<no Python frame>
[Fri Jul 17 10:52:04.927178 2020] [mpm_winnt:crit] [pid 12904:tid 588] AH00419: master_main: reate child process failed. Exiting.
mcfoi
commented
4 years ago Dear @shippo16 on Windows there is a know HACK to make it work. It's main downside is that using it will allow you to run JUST ONE wsgi Django app on one Apache2 instance as the hack is using global configuration: any wsgi app will use THE SAME virtualenv.
The hack consists in adding ONLY the following lines in httpd.conf (no other WSGI-related line must be present)
LoadModule wsgi_module "%absolute-path-to-venv-folder%/lib/site-packages/mod_wsgi/server/mod_wsgi.cp37-win_amd64.pyd" WSGIVerboseDebugging On WSGIApplicationGroup %{GLOBAL} WSGIPythonPath "%absolute-path-to-venv-folder%/Lib/site-packages;%absolute-path-to-django-project-root-folder%"
To be clear, %absolute-path-to-django-project-root-folder% is the path to the folder containing "manage.py". %absolute-path-to-venv-folder% is the path to the Virtualenv folder where all dependencies for your project have been installed
DO NOT ADD directive WSGIPythonHome -> will lead to "ModuleNotFoundError: No module named 'encodings'"
shippo16
commented
4 years ago @mcfoi Thank you very much! I'll give it a try.
shippo16
commented
4 years ago @mcfoi There are many ways around this. I was just wondering why does the mod_wsgi's loaded module not honor my virtual environment's prefixes.
GrahamDumpleton
commented
4 years ago Try again with:
pip install --no-cache-dir -U https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zipand use WSGIPythonHome as should be doing. Do not use the WSGIPythonPath fiddle.
I finally got a new Window machine setup so I could do development on it for mod_wsgi.
I think this time may have got working my workaround for problems with Python C API.
GrahamDumpleton
commented
4 years ago @shippo16 The reason there are issues is because Python broke the ability to use virtual environments when doing embedding on Windows. It was broken about 6 years ago and still isn't fixed.
Since it only affect new style virtual environments it was sort of ignored and people were told to use virtualenv instead of python -m venv. Problem is that a release of virtualenv earlier this year switched to new style virtual environment layout as well, so was broken for all solutions for creating virtual environments.
Have tried a workaround but it wasn't working. Finally worked out that also have to disable even attempting to set the virtual environment home using Python C API, as keeping that as was would normally be done meant it failed before it even got to the workaround. Does mean that mod_wsgi may now not work on older Python versions, but then no one should not be using Python 2.7 now.
shippo16
commented
4 years ago @GrahamDumpleton Thank you very much for your time!
shippo16
commented
4 years ago I just tried the zip with "--no-cache-dir". It seems to be working alright. Clean Apache error.log
m4gr4th34
commented
4 years ago none of these worked for me. i'm trying to tun a simple flask app in apache on win10. At first the Conda env did not activate, so I got numpy import issues. Then I switched to virtualenv (without knowing any of these issues this year), reinstalled python 3.7 on root, as admin, updated environment variables, carefully tried out almost every iteration of every solution offered, including the fiddle and the new wsgi install here, but cannot get past python fatal error, no module named encodings in the main apache log. I honestly do not know what else to try ...
Fatal Python error: initfsencoding: unable to load the file system codec ModuleNotFoundError: No module named 'encodings'
Current thread 0x00000628 (most recent call first): [Fri Jul 24 12:24:53.632669 2020] [mpm_winnt:crit] [pid 23352:tid 800] AH00419: master_main: create child process failed. Exiting. AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using fe80::5c29:2b89:47e6:daa1. Set the 'ServerName' directive globally to suppress this message [Fri Jul 24 12:25:39.574874 2020] [mpm_winnt:notice] [pid 14080:tid 548] AH00455: Apache/2.4.43 (Win64) mod_wsgi/4.7.1 Python/3.7 configured -- resuming normal operations [Fri Jul 24 12:25:39.575875 2020] [mpm_winnt:notice] [pid 14080:tid 548] AH00456: Apache Lounge VS16 Server built: Apr 21 2020 16:23:13 [Fri Jul 24 12:25:39.575875 2020] [core:notice] [pid 14080:tid 548] AH00094: Command line: 'C:\Apache24\bin\httpd.exe -d C:/Apache24' [Fri Jul 24 12:25:39.580878 2020] [mpm_winnt:notice] [pid 14080:tid 548] AH00418: Parent: Created child process 22660 AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using fe80::5c29:2b89:47e6:daa1. Set the 'ServerName' directive globally to suppress this message AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using fe80::5c29:2b89:47e6:daa1. Set the 'ServerName' directive globally to suppress this message Fatal Python error: initfsencoding: unable to load the file system codec ModuleNotFoundError: No module named 'encodings'
Current thread 0x0000097c (most recent call first): [Fri Jul 24 12:25:42.830007 2020] [mpm_winnt:crit] [pid 14080:tid 548] AH00419: master_main: create child process failed. Exiting.
m4gr4th34
commented
4 years ago These are the parameters I play with in various stages of comment-out in httpd.conf:
LoadFile "C:/Python/Python37/python37.dll"
WSGIPythonPath "c:/apienv3/Lib/site-packages"
Here is the wsgi file, where I recently tried the 'activate this' option to no avail:
python_home = "c:/users/me/anaconda3/envs/gpt2api"
activate_this = python_home + '/Scripts/activate_this.py' exec(open(activate_this).read(), {'file': activate_this})
import sys import logging
logging.basicConfig(stream=sys.stderr)
sys.path.insert(0, 'c:/Users/me/Python_coding_folder/ChatBots/GPT2Local/gpt-2')
from WebAPI import app as application
And finally my virtual host file:
<VirtualHost *:5000> ServerAdmin webmaster@theowner.me.com
#WSGIDaemonProcess WebAPI python-home="c:/apienv2"
# "c:/users/me/anaconda3/envs/gpt2api"
WSGIScriptAlias / "c:/Users/me/Python_coding_folder/ChatBots/GPT2Local/gpt-2/index/directory/WebAPI.wsgi"
DocumentRoot "c:/Users/me/Python_coding_folder/ChatBots/GPT2Local/gpt-2"
<Directory "c:/Users/me/Python_coding_folder/ChatBots/GPT2Local/gpt-2/index">
Require all granted
</Directory>
ErrorLog "c:/Users/me/Python_coding_folder/ChatBots/GPT2Local/gpt-2/logs/error.log"
CustomLog "c:/Users/me/Python_coding_folder/ChatBots/GPT2Local/gpt-2/logs/access.log" commonI hope it's a simple something I'm overlooking. I've been trying to get apache up all night.
m4gr4th34
commented
4 years ago many thanks in advance, for any assistance or time.
GrahamDumpleton
commented
4 years ago @m4gr4th34 Did you actually try installing mod_wsgi using:
pip install --no-cache-dir -U https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zipBut then, Anaconda Python is not normal PSF Python and they screw up many things so the directory hierarchy of the installation is different, so the adding of "/Lib/site-packages" (don't use backslashes when using in Python string) may not work as that directory doesn't exist.
Please though try that develop.zip version. Also ensure that LogLevel info, rather than err or warn is set as should hopefully output some extra debugging that may help.
Also, what directories are under:
c:/apienv3/LibThx! Still not working, but shifting. To clarify, I am trying two separate solutions: (1) using Anaconda env, and (2) using virtualenv.
1) Using Anaconda env, with the old wsgy pip install, Apache launches the website, however I got an internal error due to numpy not loading. I tried your new zipfile, installing it inside the conda environment 'gpt2api'. After installing, the website doesn't launch at all, error complaining 'fatal error, cant import encoders'. So, the new zip file is a step backward in the conda env.
2) Using virtualenv, the old wsgy setup also gave 'fatal error, can't import encoders'. after installing the new zipfile i get the same error, unfortunately.
Q: where do I set LogLevel info?
c:/apienv3 is the virtualenv. Under Lib I only have: 'site-packages'. I have not touched these directories, they are whatever python spits out when doing the pip installs in the virtualenv.
m4gr4th34
commented
4 years ago I set to LogLevel info in httpd.conf. Here's the error log (same for conda or virtualenv):
AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using fe80::61e2:230c:abec:6a74. Set the 'ServerName' directive globally to suppress this message [Fri Jul 24 15:06:56.855840 2020] [so:warn] [pid 7040:tid 144] AH01574: module wsgi_module is already loaded, skipping AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using fe80::61e2:230c:abec:6a74. Set the 'ServerName' directive globally to suppress this message [Fri Jul 24 15:06:57.728188 2020] [wsgi:info] [pid 7040:tid 144] mod_wsgi (pid=7040): Initializing Python. Fatal Python error: initfsencoding: unable to load the file system codec ModuleNotFoundError: No module named 'encodings'
Current thread 0x00005fd0 (most recent call first): [Fri Jul 24 15:06:58.990777 2020] [mpm_winnt:crit] [pid 4400:tid 148] AH00419: master_main: create child process failed. Exiting. [Fri Jul 24 15:06:58.990777 2020] [core:info] [pid 4400:tid 148] AH00096: removed PID file C:/Apache24/logs/httpd.pid (pid=4400) [Fri Jul 24 15:09:57.245595 2020] [so:warn] [pid 13012:tid 596] AH01574: module wsgi_module is already loaded, skipping AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using fe80::61e2:230c:abec:6a74. Set the 'ServerName' directive globally to suppress this message [Fri Jul 24 15:09:58.124050 2020] [mpm_winnt:notice] [pid 13012:tid 596] AH00455: Apache/2.4.43 (Win64) mod_wsgi/4.7.1 Python/3.7 configured -- resuming normal operations [Fri Jul 24 15:09:58.124050 2020] [mpm_winnt:notice] [pid 13012:tid 596] AH00456: Apache Lounge VS16 Server built: Apr 21 2020 16:23:13 [Fri Jul 24 15:09:58.124050 2020] [core:notice] [pid 13012:tid 596] AH00094: Command line: 'C:\Apache24\bin\httpd.exe -d C:/Apache24' [Fri Jul 24 15:09:58.129056 2020] [mpm_winnt:notice] [pid 13012:tid 596] AH00418: Parent: Created child process 11688 [Fri Jul 24 15:09:58.505057 2020] [so:warn] [pid 11688:tid 524] AH01574: module wsgi_module is already loaded, skipping AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using fe80::61e2:230c:abec:6a74. Set the 'ServerName' directive globally to suppress this message [Fri Jul 24 15:09:59.384908 2020] [so:warn] [pid 11688:tid 524] AH01574: module wsgi_module is already loaded, skipping AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using fe80::61e2:230c:abec:6a74. Set the 'ServerName' directive globally to suppress this message [Fri Jul 24 15:10:00.248065 2020] [wsgi:info] [pid 11688:tid 524] mod_wsgi (pid=11688): Initializing Python. Fatal Python error: initfsencoding: unable to load the file system codec ModuleNotFoundError: No module named 'encodings'
Current thread 0x00000a34 (most recent call first): [Fri Jul 24 15:10:01.549250 2020] [mpm_winnt:crit] [pid 13012:tid 596] AH00419: master_main: create child process failed. Exiting. [Fri Jul 24 15:10:01.549250 2020] [core:info] [pid 13012:tid 596] AH00096: removed PID file C:/Apache24/logs/httpd.pid (pid=13012)
m4gr4th34
commented
4 years ago btw, thanks for helping out. the package is very cool; such a shame that virtualenv has so many issues that you need to dance around it seems. it's very cool of you.
GrahamDumpleton
commented
4 years ago Your config is mucked up somehow as you have this message:
[Fri Jul 24 15:09:59.384908 2020] [so:warn] [pid 11688:tid 524] AH01574: module wsgi_module is already loaded, skippingThis means you have two LoadModule lines for wsgi_module, or are including it twice from an include file. You may want to resolve that and ensure loading correct module file.
If using numpy, you also must use:
WSGIApplicationGroup "%{GLOBAL}"I would suggest you go back to normal PSF Python and tell me what the numpy error was and we sort that out rather than trying to switch to Anaconda Python.
To verify the config, show what you are setting for:
LoadFile "C:/Python/Python37/python37.dll"
LoadModule wsgi_module "c:/apienv3/lib/site-packages/mod_wsgi/server/mod_wsgi.cp37-win_amd64.pyd"
WSGIPythonHome "c:/apienv3"
# WSGIPythonPath "c:/apienv3/Lib/site-packages"when you do that.
m4gr4th34
commented
4 years ago I rebooted the system and the 'config' error message went away, but i still get python load errors. To clarify, Anaconda Python gave me the numpy error. Normal PSF Python never got me past python load error. But I agree, let's stick with PSF Python since that is more standard procedure. I added the line WSGIApplicationGroup "%{GLOBAL}" to httpd.conf, as follows:
LoadFile 'C:/Python/Python37/python37.dll' LoadModule wsgi_module 'c:/apienv3/lib/site-packages/mod_wsgi/server/mod_wsgi.cp37-win_amd64.pyd' WSGIApplicationGroup "%{GLOBAL}" WSGIPythonHome 'c:/apienv3'
I also tried to comment out 'home' and use 'path', but still the same error. I have been trying every iteration and combination of directory paths and parameters, but no luck.
Error log:
AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using fe80::61e2:230c:abec:6a74. Set the 'ServerName' directive globally to suppress this message [Fri Jul 24 18:16:18.122240 2020] [mpm_winnt:notice] [pid 16572:tid 800] AH00455: Apache/2.4.43 (Win64) mod_wsgi/4.7.1 Python/3.7 configured -- resuming normal operations [Fri Jul 24 18:16:18.124241 2020] [mpm_winnt:notice] [pid 16572:tid 800] AH00456: Apache Lounge VS16 Server built: Apr 21 2020 16:23:13 [Fri Jul 24 18:16:18.124241 2020] [core:notice] [pid 16572:tid 800] AH00094: Command line: 'C:\Apache24\bin\httpd.exe -d C:/Apache24' [Fri Jul 24 18:16:18.130243 2020] [mpm_winnt:notice] [pid 16572:tid 800] AH00418: Parent: Created child process 13412 AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using fe80::61e2:230c:abec:6a74. Set the 'ServerName' directive globally to suppress this message AH00558: httpd.exe: Could not reliably determine the server's fully qualified domain name, using fe80::61e2:230c:abec:6a74. Set the 'ServerName' directive globally to suppress this message [Fri Jul 24 18:16:20.274324 2020] [wsgi:info] [pid 13412:tid 744] mod_wsgi (pid=13412): Initializing Python. Fatal Python error: initfsencoding: unable to load the file system codec ModuleNotFoundError: No module named 'encodings'
Current thread 0x00002534 (most recent call first): [Fri Jul 24 18:16:21.383571 2020] [mpm_winnt:crit] [pid 16572:tid 800] AH00419: master_main: create child process failed. Exiting. [Fri Jul 24 18:16:21.384571 2020] [core:info] [pid 16572:tid 800] AH00096: removed PID file C:/Apache24/logs/httpd.pid (pid=16572)
m4gr4th34
commented
4 years ago When I was using Anaconda Python, this is the numpy error that I got (just fyi. I switched to PSF Python because I thought it would be simpler to solve the problem -- boy was i wrong about that...):
Fri Jul 24 13:01:08.695613 2020] [wsgi:error] [pid 12672:tid 1376] [client 127.0.0.1:59089] \r
[Fri Jul 24 13:01:08.695613 2020] [wsgi:error] [pid 12672:tid 1376] [client 127.0.0.1:59089] During handling of the above exception, another exception occurred:\r
[Fri Jul 24 13:01:08.695613 2020] [wsgi:error] [pid 12672:tid 1376] [client 127.0.0.1:59089] \r
[Fri Jul 24 13:01:08.695613 2020] [wsgi:error] [pid 12672:tid 1376] [client 127.0.0.1:59089] Traceback (most recent call last):\r
[Fri Jul 24 13:01:08.695613 2020] [wsgi:error] [pid 12672:tid 1376] [client 127.0.0.1:59089] File "C:/Users/irfan/Python_coding_folder/ChatBots/GPT2Local/gpt-2/index/directory/WebAPI.wsgi", line 16, in
I an getting this error: [wsgi:info] [..] mod_wsgi (pid=4444): Python home E:\venvs\meteoapi. [wsgi:info] [..] mod_wsgi (pid=4444): Initializing Python. Fatal Python error: initfsencoding: unable to load the file system codec ModuleNotFoundError: No module named 'encodings'
What I did is the following, while in Windows Server 2012R2:
Install Python 3.7 32bit with symbols Install MS C++ compiler VC14 Install Apache 2.4.41 VC14 32bit in default
c:\Apache24Create virtualenv inE:\venvs\meteoapiActivate virtualenv runningE:\venvs\meteoapi\Scripts\activate.batinstall mod_wsgi-express with:pip install mod_wsgisThis created 'mod_wsgi.cp37-win32.pyd' in e:/venvs/meteoapi/lib/ Run from activated venvmod_wsgi-express module-configCopy output to httpd.conf to load mod_wsgi in Apache.LoadModule wsgi_module "e:/venvs/meteoapi/lib/site-packages/mod_wsgi/server/mod_wsgi.cp37-win32.pyd"WSGIPythonHome "e:/venvs/meteoapi"Start Apache2.4 with:c:\Apache24\bin\httpd.exeApache doesn't start but error.logs shows what reported above:
Fatal Python error: initfsencoding: unable to load the file system codec ModuleNotFoundError: No module named 'encodings'