search

unbit / uwsgi

uWSGI application server container
http://projects.unbit.it/uwsgi
Other
3.45k stars 691 forks source link

uwsgi should exit when app fails to load on startup #622

Closed benbeanfield closed 10 years ago

benbeanfield commented 10 years ago

I have a python app that I serve using uwsgi. During the initial startup, I'd like the master process to exit if the worker processes (app instances) fail to initialize.

uwsgi configuration:

master = true
lazy-apps = true
need-app = true

If I set master = false or lazy-apps = false then uwsgi behaves as I'd expect. Unfortunately, I require lazy-apps, and I'm hesitant to set master=false because the uwsgi documentation advises against this except when using Emperor Mode.

I believe that this issue was supposed to be resolved in #557, but I'm still experiencing this issue with the current git master head.

unbit commented 10 years ago

Can't reproduce it:

uwsgi --wsgi badapp --lazy-apps --need-app --socket :0 --master --processes 8

maybe you have some other option changing the behaviour ?

benbeanfield commented 10 years ago

I've found that this issue only manifests when I use "--ini-paste-logged" to initialize my Pyramid application.

In this example, the application continuously restarts when it fails to import the (intentionally missing) "six" dependency, which raises a DistributionNotFound exception during app startup:

uwsgi --ini-paste-logged staging.ini --lazy-apps --need-app --socket :0 --master --processes 2

When using "--wsgi staging" ("staging.py" is a small module used to boostrap the Pyramid app for "--wsgi"), uWSGI exits when the DistributionNotFound exception is raised - which is the expected behaviour:

uwsgi --wsgi staging  --lazy-apps --need-app  --socket :0 --master --processes 2

staging.py:

from pyramid.paster import get_app, setup_logging
ini_path = './staging.ini'
setup_logging(ini_path)
application = get_app(ini_path, 'main')

staging.ini contains an empty [uwsgi] section as well as minimal app-specific configuration:

[app:main]
use = egg:foo#main
sqlalchemy.url = postgresql://redacted/redacted

[uwsgi]

Here's a log showing the continuous restarts, which only occurs when uWSGI is invoked with "--ini-paste-logged":

uwsgi --ini-paste-logged staging.ini --lazy-apps --need-app --socket :0 --master --processes 2
[uWSGI] getting INI configuration from staging.ini
*** WARNING: Can't find section "uwsgi" in INI configuration file staging.ini ***
*** Starting uWSGI 2.0.5 (64bit) on [Tue May 13 10:44:09 2014] ***
compiled with version: 4.4.7 20120313 (Red Hat 4.4.7-4) on 09 May 2014 14:38:00
os: Linux-2.6.32-431.11.2.el6.x86_64 #1 SMP Tue Mar 25 11:15:18 CDT 2014
nodename: api.staging.bar.example.com
machine: x86_64
clock source: unix
detected number of CPU cores: 2
current working directory: /opt/example/foo/app
detected binary path: /opt/example/foo/venvs/foo/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** 
your processes number limit is 30510
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address :49113 (port auto-assigned) fd 3
Python version: 2.7.3 (default, Nov  5 2012, 17:26:19)  [GCC 4.4.5 20110214 (Red Hat 4.4.5-6)]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x141ac40
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 218304 bytes (213 KB) for 2 cores
*** Operational MODE: preforking ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 925)
spawned uWSGI worker 1 (pid: 926, cores: 1)
spawned uWSGI worker 2 (pid: 927, cores: 1)
Loading paste environment: config:/opt/example/foo/app/staging.ini
Loading paste environment: config:/opt/example/foo/app/staging.ini
Traceback (most recent call last):
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/script/util/logging_config.py", line 76, in fileConfig
Traceback (most recent call last):
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/script/util/logging_config.py", line 76, in fileConfig
    formatters = _create_formatters(cp)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/script/util/logging_config.py", line 108, in _create_formatters
    flist = cp.get("formatters", "keys")
      File "/usr/lib64/python2.7/ConfigParser.py", line 607, in get
formatters = _create_formatters(cp)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/script/util/logging_config.py", line 108, in _create_formatters
    flist = cp.get("formatters", "keys")
  File "/usr/lib64/python2.7/ConfigParser.py", line 607, in get
    raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'formatters'
    raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'formatters'
Traceback (most recent call last):
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 271, in loadobj
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
Traceback (most recent call last):
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 320, in _loadconfig
    return loadobj(APP, uri, name=name, **kw)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 271, in loadobj
    return loader.get_context(object_type, name, global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 454, in get_context
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    section)
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 476, in _context_from_use
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 320, in _loadconfig
    return loader.get_context(object_type, name, global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 454, in get_context
    object_type, name=use, global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 406, in get_context
    section)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 476, in _context_from_use
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 328, in _loadegg
    object_type, name=use, global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 406, in get_context
    return loader.get_context(object_type, name, global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 620, in get_context
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 328, in _loadegg
    object_type, name=name)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 640, in find_egg_entry_point
    return loader.get_context(object_type, name, global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 620, in get_context
    pkg_resources.require(self.spec)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/pkg_resources.py", line 728, in require
    object_type, name=name)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 640, in find_egg_entry_point
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
    pkg_resources.require(self.spec)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/pkg_resources.py", line 728, in require
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: six
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: six
OOPS ! failed loading app in worker 1 (pid 926) :( trying again...
Respawned uWSGI worker 1 (new pid: 928)
OOPS ! failed loading app in worker 2 (pid 927) :( trying again...
Respawned uWSGI worker 2 (new pid: 929)
Loading paste environment: config:/opt/example/foo/app/staging.ini
Loading paste environment: config:/opt/example/foo/app/staging.ini
Traceback (most recent call last):
Traceback (most recent call last):
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/script/util/logging_config.py", line 76, in fileConfig
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/script/util/logging_config.py", line 76, in fileConfig
    formatters = _create_formatters(cp)
    formatters = _create_formatters(cp)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/script/util/logging_config.py", line 108, in _create_formatters
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/script/util/logging_config.py", line 108, in _create_formatters
    flist = cp.get("formatters", "keys")
    flist = cp.get("formatters", "keys")
  File "/usr/lib64/python2.7/ConfigParser.py", line 607, in get
  File "/usr/lib64/python2.7/ConfigParser.py", line 607, in get
    raise NoSectionError(section)
    raise NoSectionError(section)
ConfigParser.NoSectionErrorConfigParser: .NoSectionErrorNo section: 'formatters'
: No section: 'formatters'
Traceback (most recent call last):
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 271, in loadobj
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 320, in _loadconfig
    return loader.get_context(object_type, name, global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 454, in get_context
    section)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 476, in _context_from_use
    object_type, name=use, global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 406, in get_context
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
Traceback (most recent call last):
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 328, in _loadegg
    return loadobj(APP, uri, name=name, **kw)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 271, in loadobj
    return loader.get_context(object_type, name, global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 620, in get_context
    object_type, name=name)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 640, in find_egg_entry_point
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    pkg_resources.require(self.spec)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/pkg_resources.py", line 728, in require
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 320, in _loadconfig
    return loader.get_context(object_type, name, global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 454, in get_context
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
    section)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 476, in _context_from_use
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: six
    object_type, name=use, global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 406, in get_context
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 328, in _loadegg
    return loader.get_context(object_type, name, global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 620, in get_context
    object_type, name=name)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 640, in find_egg_entry_point
    pkg_resources.require(self.spec)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/pkg_resources.py", line 728, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: six
OOPS ! failed loading app in worker 1 (pid 928) :( trying again...
worker respawning too fast !!! i have to sleep a bit (2 seconds)...
Respawned uWSGI worker 1 (new pid: 930)
OOPS ! failed loading app in worker 2 (pid 929) :( trying again...
Respawned uWSGI worker 2 (new pid: 931)
Loading paste environment: config:/opt/example/foo/app/staging.ini
Loading paste environment: config:/opt/example/foo/app/staging.ini
Traceback (most recent call last):
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/script/util/logging_config.py", line 76, in fileConfig
    formatters = _create_formatters(cp)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/script/util/logging_config.py", line 108, in _create_formatters
Traceback (most recent call last):
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/script/util/logging_config.py", line 76, in fileConfig
    flist = cp.get("formatters", "keys")
  File "/usr/lib64/python2.7/ConfigParser.py", line 607, in get
    formatters = _create_formatters(cp)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/script/util/logging_config.py", line 108, in _create_formatters
    flist = cp.get("formatters", "keys")
  File "/usr/lib64/python2.7/ConfigParser.py", line 607, in get
    raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'formatters'
    raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'formatters'
Traceback (most recent call last):
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
Traceback (most recent call last):
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 271, in loadobj
    return loadobj(APP, uri, name=name, **kw)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 271, in loadobj
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 320, in _loadconfig
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 320, in _loadconfig
    return loader.get_context(object_type, name, global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 454, in get_context
    return loader.get_context(object_type, name, global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 454, in get_context
    section)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 476, in _context_from_use
    section)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 476, in _context_from_use
    object_type, name=use, global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 406, in get_context
    object_type, name=use, global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 406, in get_context
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 328, in _loadegg
    return loader.get_context(object_type, name, global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 620, in get_context
    object_type, name=name)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 640, in find_egg_entry_point
    global_conf=global_conf)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 328, in _loadegg
    return loader.get_context(object_type, name, global_conf)
    pkg_resources.require(self.spec)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 620, in get_context
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/pkg_resources.py", line 728, in require
    object_type, name=name)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 640, in find_egg_entry_point
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
    pkg_resources.require(self.spec)
    raise DistributionNotFound(req)
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/pkg_resources.py", line 728, in require
pkg_resources.DistributionNotFound: six
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/example/foo/venvs/foo/lib/python2.7/site-packages/pkg_resources.py", line 626, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: six