search

WebarchivCZ / Seeder

Seeder - Czech webarchive curating tool and public site
MIT License
15 stars 2 forks source link

manage.py migrate fails #349

Closed westfood closed 7 years ago

westfood commented 7 years ago

Ahoj,

v nové verzi mi to hází tohle:

(seeder) bash-4.2$ ./Seeder/manage.py migrate
Traceback (most recent call last):
  File "./Seeder/manage.py", line 17, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/core/management/__init__.py", line 359, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/core/management/base.py", line 294, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/core/management/base.py", line 342, in execute
    self.check()
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/core/management/base.py", line 374, in check
    include_deployment_checks=include_deployment_checks,
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/core/management/commands/migrate.py", line 62, in _run_checks
    issues.extend(super(Command, self)._run_checks(**kwargs))
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/core/management/base.py", line 361, in _run_checks
    return checks.run_checks(**kwargs)
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/core/checks/registry.py", line 81, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/core/checks/urls.py", line 14, in check_url_config
    return check_resolver(resolver)
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/core/checks/urls.py", line 24, in check_resolver
    for pattern in resolver.url_patterns:
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/urls/resolvers.py", line 313, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/opt/virtualenv/seeder/lib/python3.4/site-packages/django/urls/resolvers.py", line 306, in urlconf_module
    return import_module(self.urlconf_name)
  File "/opt/virtualenv/seeder/lib64/python3.4/importlib/__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1471, in exec_module
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "/opt/Seeder/Seeder/urls.py", line 20, in <module>
    from www import views as www_views
  File "/opt/Seeder/Seeder/www/views.py", line 39
    **super().get_context_data()
     ^
SyntaxError: invalid syntax
Visgean commented 7 years ago

Mas stary python - ja mam Python 3.5.2

westfood commented 7 years ago

Hele, doteď se jelo na 3.4, 3.5 není v repozitářích CentOS, 3.5 musím kompilovat - je to stopro nutné?

Visgean commented 7 years ago

jsi si jisty, ze to tam neni? https://www.softwarecollections.org/en/scls/rhscl/rh-python35/ muzu to fixnout docela jednoduse ale lepsi by bylo mit 3.5... Ale jestli neni tak neni.

westfood commented 7 years ago

Pěkný repo, testnu to.

westfood commented 7 years ago

Hele ještě jeden dotaz, jak je to s UWSGI a virtualenv?

UWSGI běží v Python verzi: 3.4.3 Virtualenv má binárky pro Python: 3.5

To je cajk nebo ne?

*** has_emperor mode detected (fd: 7) ***
[uWSGI] getting INI configuration from seeder.ini
*** Starting uWSGI 2.0.12 (64bit) on [Wed Feb 15 23:16:06 2017] ***
compiled with version: 4.8.5 20150623 (Red Hat 4.8.5-4) on 02 January 2016 19:52:54
os: Linux-3.10.0-327.13.1.el7.x86_64 #1 SMP Thu Mar 31 16:04:38 UTC 2016
nodename: war.webarchiv.cz
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 8
current working directory: /etc/uwsgi.d
writing pidfile to /opt/Seeder/seeder.pid
detected binary path: /usr/sbin/uwsgi
your processes number limit is 31212
your memory page size is 4096 bytes
 *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /opt/Seeder/seeder.sock fd 3
Python version: 3.4.3 (default, Jan 26 2016, 02:25:35)  [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)]
Set PythonHome to /opt/virtualenv/seeder
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
Wed Feb 15 23:16:07 2017 - [emperor] curse the uwsgi instance seeder.ini (pid: 5460)
Wed Feb 15 23:16:07 2017 - [emperor] removed uwsgi instance seeder.ini
westfood commented 7 years ago

tak malý posun..

--- no python application found, check your startup logs for errors ---
[pid: 25886|app: -1|req: -1/1] 66.249.64.112 () {34 vars in 508 bytes} [Thu Feb 16 13:13:21 2017] GET /robots.txt => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
announcing my loyalty to the Emperor...
Thu Feb 16 13:13:21 2017 - [emperor] vassal seeder.ini is now loyal
--- no python application found, check your startup logs for errors ---
[pid: 25885|app: -1|req: -1/2] 66.249.64.104 () {36 vars in 623 bytes} [Thu Feb 16 13:13:21 2017] GET /wayback/*/http://www.uur.cz => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
announcing my loyalty to the Emperor...
Thu Feb 16 13:13:21 2017 - [emperor] vassal seeder.ini is now loyal
--- no python application found, check your startup logs for errors ---
[pid: 25886|app: -1|req: -1/3] 195.113.132.25 () {42 vars in 1149 bytes} [Thu Feb 16 13:13:24 2017] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
*** Starting uWSGI 2.0.12 (64bit) on [Thu Feb 16 13:10:38 2017] ***
compiled with version: 4.8.5 20150623 (Red Hat 4.8.5-4) on 02 January 2016 19:52:54
os: Linux-3.10.0-327.13.1.el7.x86_64 #1 SMP Thu Mar 31 16:04:38 UTC 2016
nodename: war.webarchiv.cz
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 8
current working directory: /
writing pidfile to /run/uwsgi/uwsgi.pid
detected binary path: /usr/sbin/uwsgi
setgid() to 987
setuid() to 991
your processes number limit is 31212
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
*** starting uWSGI Emperor ***
your mercy for graceful operations on workers is 60 seconds
*** Operational MODE: no-workers ***
spawned uWSGI master process (pid: 25882)
*** Stats server enabled on /run/uwsgi/stats.sock fd: 7 ***
*** has_emperor mode detected (fd: 7) ***
[uWSGI] getting INI configuration from seeder.ini
*** Starting uWSGI 2.0.12 (64bit) on [Thu Feb 16 13:10:38 2017] ***
compiled with version: 4.8.5 20150623 (Red Hat 4.8.5-4) on 02 January 2016 19:52:54
os: Linux-3.10.0-327.13.1.el7.x86_64 #1 SMP Thu Mar 31 16:04:38 UTC 2016
nodename: war.webarchiv.cz
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 8
current working directory: /etc/uwsgi.d
writing pidfile to /opt/Seeder/seeder.pid
detected binary path: /usr/sbin/uwsgi
your processes number limit is 31212
your memory page size is 4096 bytes
 *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /opt/Seeder/seeder.sock fd 3
Python version: 3.4.3 (default, Jan 26 2016, 02:25:35)  [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)]
PEP 405 virtualenv detected: /opt/virtualenv/seeder
Set PythonHome to /opt/virtualenv/seeder
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1350540
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 509376 bytes (497 KB) for 6 cores
*** Operational MODE: preforking ***
added /opt/Seeder/Seeder/ to pythonpath.
Traceback (most recent call last):
  File "/opt/Seeder/Seeder/wsgi.py", line 13, in <module>
    from django.core.wsgi import get_wsgi_application
ImportError: No module named 'django'
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 25884)
Thu Feb 16 13:10:38 2017 - [emperor] vassal seeder.ini has been spawned
spawned uWSGI worker 1 (pid: 25885, cores: 1)
spawned uWSGI worker 2 (pid: 25886, cores: 1)
unable to stat() /opt/Seeder/reload.touch, events will be triggered as soon as the file is created
Thu Feb 16 13:10:38 2017 - [emperor] vassal seeder.ini is ready to accept requests
westfood commented 7 years ago

uwsgi mi píše:

(seeder) [root@war Seeder]# tail -f /var/log/seeder/uwsgi.log
--- no python application found, check your startup logs for errors ---
[pid: 8522|app: -1|req: -1/16] 195.113.132.25 () {42 vars in 1157 bytes} [Thu Feb 16 15:59:10 2017] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
westfood commented 7 years ago

imho jde o tohle: ImportError: No module named 'django'

Ale moc nevím co si s tím počít

*** Operational MODE: preforking ***
added /opt/Seeder/Seeder/ to pythonpath.
Traceback (most recent call last):
  File "/opt/Seeder/Seeder/wsgi.py", line 13, in <module>
    from django.core.wsgi import get_wsgi_application
ImportError: No module named 'django'
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
westfood commented 7 years ago

Jestli je vyjebaný virtualenv nebo uwsgi

westfood commented 7 years ago

např. 

(seeder) [root@war Seeder]# Seeder/manage.py migrate
System check identified some issues:

WARNINGS:
?: (mysql.W002) MySQL Strict Mode is not set for database connection 'legacy_seeder'
        HINT: MySQL's Strict Mode fixes many data integrity problems in MySQL, such as data truncation upon insertion, by escalating warnings into errors. It is strongly recommended you activate it. See: https://docs.djangoproject.com/en/1.10/ref/databases/#mysql-sql-mode
source.Source.keywords: (fields.W340) null has no effect on ManyToManyField.
www.TopicCollection.sources: (fields.W340) null has no effect on ManyToManyField.
Operations to perform:
  Apply all migrations: admin, auth, authtoken, blacklists, comments, contenttypes, contracts, harvests, legacy_db, publishers, qa, reversion, sessions, source, thumbnail, voting, www
Running migrations:
  No migrations to apply.
Visgean commented 7 years ago

hmm, imho v tom wsgi nepouzivas virtualenv.

Visgean commented 7 years ago

resp rozhodne nepouzivas spravny, doporucuji ten stary smazat a vytvorit novy.

Visgean commented 7 years ago
westfood commented 7 years ago

Hele, je novej ve 3.5.

čt 16. 2. 2017 v 23:22 odesílatel Rudolf K. westfood@gmail.com napsal:

čt 16. 2. 2017 v 20:26 odesílatel Visgean Skeloru < notifications@github.com> napsal:

  • by bylo fajn vytvorit ten novy pomoci python3.5

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/WebarchivCZ/Seeder/issues/349#issuecomment-280432413, or mute the thread https://github.com/notifications/unsubscribe-auth/AATQLKHIt8K5KScLMfIAgbXDA9ENw3NOks5rdKLdgaJpZM4MBjCy .

Visgean commented 7 years ago

a jakou mas konfiguraci wsgi?

westfood commented 7 years ago 
(seeder) [root@war Seeder]# cat /etc/uwsgi.ini
[uwsgi]
uid = uwsgi
gid = uwsgi
pidfile = /run/uwsgi/uwsgi.pid
emperor = /etc/uwsgi.d
stats = /run/uwsgi/stats.sock
cap = setgid,setuid
logto = /var/log/seeder/uwsgi.log
(seeder) [root@war Seeder]# cat /etc/uwsgi.d/seeder.ini
[uwsgi]
virtualenv      = /opt/virtualenv/seeder
pythonpath      = /opt/Seeder/Seeder
plugins         = python3
master          = 1
pidfile         = /opt/Seeder/seeder.pid
touch-reload    = /opt/Seeder/reload.touch
socket          = /opt/Seeder/seeder.sock
chmod-socket    = 777
cheaper-algo    = spare
cheaper         = 2
cheaper-initial = 2
cheaper-step    = 1
processes       = 6
vacuum          = 1
harakiri        = 60
wsgi-file       = /opt/Seeder/Seeder/wsgi.py
env             = PATH=/opt/rh/rh-python35/root/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
env             = LD_LIBRARY_PATH=/opt/rh/rh-python35/root/usr/lib64
env             = XDG_DATA_DIRS=/opt/rh/rh-python35/root/usr/share:/usr/local/share:/usr/share
env             = PKG_CONFIG_PATH=/opt/rh/rh-python35/root/usr/lib64/pkgconfig
scl enable rh-python35 bash
virtualenv seeder
případně:
virtualenv -p /opt/rh/rh-python35/root/usr/bin/python seeder

mám pocit, že jsem zkoušel i 
python -m venv seeder

když virtualenv aktivuji:

(seeder) [root@war seeder]# python -V
python: error while loading shared libraries: libpython3.5m.so.rh-python35-1.0: cannot open shared object file: No such file or directory

Jediné jak to funguje je:

(seeder) [root@war seeder]# source /opt/rh/rh-python35/enable
(seeder) [root@war seeder]# python -V
Python 3.5.1

Proto jsem navíc dodal ty env, do seeder.ini pro uwsgi.

Visgean commented 7 years ago

http://modwsgi.readthedocs.io/en/develop/user-guides/installation-issues.html hmm, myslim ze je problem, ze to wsgi neni zkompilovane pomoci pythonu3.5, ale ok, asi to bude nejjednodusi dat zpatky na python3.4

westfood commented 7 years ago

tak můžu ho zkusit buildnout s 3.5

Visgean commented 7 years ago

jak chces, pro me to neni zas takovy problem zmenit a vypada to ze je to komplikovany proces rozchodit pthon 3.5

westfood commented 7 years ago

testnu a uvidíme

westfood commented 7 years ago

Ahoj, bavil jsem se s Járou. Můžeš to hodit na 3.4? Zkusím paralerně připravit prostředí kde to půjde ve 3.5, ale teď bychom potřebovali testovat, tak to bude nejrychlejší.

Visgean commented 7 years ago

jasne, udelam to dneska.