Database connection closes unexpectedly

[Ka0o0]

Hi,

I'm running acme2certifier with Django and uwsgi behind a NGINX reverse proxy. Here are my server configuration files:

Django Settings

``` """ Django settings for acme2certifier project. Generated by 'django-admin startproject' using Django 1.11.15. For more information on this file, see https://docs.djangoproject.com/en/1.11/topics/settings/ For the full list of settings and their values, see https://docs.djangoproject.com/en/1.11/ref/settings/ """ import os import sys # Build paths inside the project like this: os.path.join(BASE_DIR, ...) BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) def get_env_value_or_exit(var): if var not in os.environ: print(f"Error: The following environment variable is missing: {var}") sys.exit(1) else: return os.environ[var] # Quick-start development settings - unsuitable for production # See https://docs.djangoproject.com/en/1.11/howto/deployment/checklist/ # SECURITY WARNING: don't run with debug turned on in production! DEBUG = False ALLOWED_HOSTS = ['127.0.0.1','*'] # Application definition INSTALLED_APPS = [ 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'acme_srv' ] MIDDLEWARE = [ 'django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', # 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', ] ROOT_URLCONF = 'acme2certifier.urls' TEMPLATES = [ { 'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': { 'context_processors': [ 'django.template.context_processors.debug', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', ], }, }, ] WSGI_APPLICATION = 'acme2certifier.wsgi.application' # Database # https://docs.djangoproject.com/en/1.11/ref/settings/#databases DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': get_env_value_or_exit('DB_NAME'), 'USER': get_env_value_or_exit('DB_USER'), 'PASSWORD': get_env_value_or_exit('DB_PASSWORD'), 'HOST': get_env_value_or_exit('DB_HOST'), 'PORT': get_env_value_or_exit('DB_PORT'), } } # Password validation # https://docs.djangoproject.com/en/1.11/ref/settings/#auth-password-validators AUTH_PASSWORD_VALIDATORS = [ { 'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator', }, { 'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator', }, { 'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator', }, { 'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator', }, ] # Internationalization # https://docs.djangoproject.com/en/1.11/topics/i18n/ LANGUAGE_CODE = 'en-us' TIME_ZONE = 'UTC' USE_I18N = False USE_L10N = True USE_TZ = False # Static files (CSS, JavaScript, Images) # https://docs.djangoproject.com/en/1.11/howto/static-files/ STATIC_URL = '/static/' # SECURITY WARNING: keep the secret key used in production secret! SECRET_KEY = get_env_value_or_exit('SECRET_KEY') DEFAULT_AUTO_FIELD='django.db.models.AutoField' ```

UWSGI Configuration File

``` [uwsgi] strict = true module = acme2certifier_wsgi:application master = true enable-threads = true vacuum = true single-interpreter = true need-app = true processes = 1 threads = 5 uid = www-data socket = :9090 die-on-term = true ```

NGINX Sever Section

``` server { listen 443 default_server ssl; listen [::]:443 default_server ssl; server_name acme01.company.com; ssl_certificate /data/tls_certificates/tls.crt; ssl_certificate_key /data/tls_certificates/tls.key; # first 5 requests go trough instantly 5more requests evey 100ms limit_req zone=ip burst=10; # delay=5; server_name _; location = favicon.ico { access_log off; log_not_found off; } location / { access_log off; include uwsgi_params; uwsgi_pass "acme2certifier:9090"; if ($request_method = "HEAD" ) { add_header Content-length 0; # add_header Transfer-Encoding identity; } } } ```

My problem is that after running for a couple of hours, acme2certifier seems to loose connection to the PostgreSQL database. Here is the error message:

acme2certifier database error in Nonce.generate_and_add(): server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.

A restart of the service solves the problem so I'm pretty sure that this is not a problem with the DB itself. I'm a bit unsure if there is some sort of keep-alive mechanism not working properly and which component might be configured wrongly. I've activated threading support on UWSGI and limited the number of processes to 1, so might this be a problem? Anyway, any help is highly appreciated.

BR Kai

[grindsa]

Hi,

What are you trying to achieve by enabling threading support/limiting the uwsgi process? (sorry I am not really an expert in this topic). Can you please share the UWSGI configuration that i can try to replicate this issue on side?

BR G.

[Ka0o0]

The communication with our PKI takes some time when issuing certificates. I wanted to serve multiple requests in parallel so I've enabled threading here. I've read, however, that uWSGI threading might create some issues with database connections. I'll test if disabling threading fixes the problem.