jazzband / django-nose

Django test runner using nose
http://pypi.python.org/pypi/django-nose
BSD 3-Clause "New" or "Revised" License
882 stars 234 forks source link

Tests failing only when using django-nose and Django 2.X #307

Closed dlareau closed 4 years ago

dlareau commented 4 years ago

EDIT: See update in second post, I've shrunk the example code down quite a bit and I'm still having this issue.

Double edit: Fix proposed and pull request submitted, waiting on confirmation that the fix is valid.

I ran into this while trying to upgrade from Django 1.11 to 2.2

Previously on Django 1.11 I had django-nose installed and all the tests ran without issue. When I tried to upgrade to Django 2, every test that touches the database in any way now throws the error:

======================================================================
ERROR: Test the index page
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/puzzlehunt/puzzlehunt_server/huntserver/tests.py", line 115, in test_index
    response = get_and_check_page(self, 'huntserver:index', 200)
  File "/opt/puzzlehunt/puzzlehunt_server/huntserver/tests.py", line 42, in get_and_check_page
    response = test.client.get(reverse(page, kwargs=args))
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/test/client.py", line 517, in get
    response = super().get(path, data=data, secure=secure, **extra)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/test/client.py", line 332, in get
    return self.generic('GET', path, secure=secure, **r)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/test/client.py", line 404, in generic
    return self.request(**r)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/test/client.py", line 485, in request
    raise exc_value
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 35, in inner
    response = get_response(request)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 128, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 126, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/puzzlehunt/puzzlehunt_server/huntserver/info_views.py", line 17, in index
    curr_hunt = Hunt.objects.get(is_current_hunt=True)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 397, in get
    num = len(clone)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 254, in __len__
    self._fetch_all()
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 1179, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 54, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1066, in execute_sql
    cursor.close()
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/MySQLdb/cursors.py", line 85, in close
    while self.nextset():
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/MySQLdb/cursors.py", line 173, in nextset
    nr = db.next_result()
_mysql_exceptions.OperationalError: (2006, '')
-------------------- >> begin captured logging << --------------------
django.request: ERROR: Internal Server Error: /
Traceback (most recent call last):
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1063, in execute_sql
    cursor.execute(sql, params)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 80, in _execute
    self.db.validate_no_broken_transaction()
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/backends/base/base.py", line 437, in validate_no_broken_transaction
    "An error occurred in the current transaction. You can't "
django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 35, in inner
    response = get_response(request)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 128, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 126, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/puzzlehunt/puzzlehunt_server/huntserver/info_views.py", line 17, in index
    curr_hunt = Hunt.objects.get(is_current_hunt=True)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 397, in get
    num = len(clone)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 254, in __len__
    self._fetch_all()
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 1179, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/query.py", line 54, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1066, in execute_sql
    cursor.close()
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/MySQLdb/cursors.py", line 85, in close
    while self.nextset():
  File "/opt/puzzlehunt/puzzlehunt_server/venv/lib/python3.5/site-packages/MySQLdb/cursors.py", line 173, in nextset
    nr = db.next_result()
_mysql_exceptions.OperationalError: (2006, '')
--------------------- >> end captured logging << ---------------------

This only happens on Django 2.X, and only when django-nose's test runner is enabled. Removing the line

TEST_RUNNER = 'django_nose.NoseTestSuiteRunner'

from settings.py fixes the issue.

These errors also don't seem to come up at all outside of unit testing (which I guess makes sense if the issue is somehow related to django-nose).

Here is an example test from the test.py file:

from django.test import TestCase
from django.urls import reverse
from huntserver import models

def get_and_check_page(test, page, code, args={}):
    response = test.client.get(reverse(page, kwargs=args))
    test.assertEqual(response.status_code, code)
    return response

class InfoTests(TestCase):
    fixtures = ["basic_hunt"]

    def test_index(self):
        "Test the index page"
        response = get_and_check_page(self, 'huntserver:index', 200)
        self.assertTrue(isinstance(response.context['curr_hunt'], models.Hunt))

I'm seeing these issues locally locally on Debian 9 (Stretch), but it also seems to fail in the same way on travis-ci (Which appears to be ubuntu 16.04): https://travis-ci.org/dlareau/puzzlehunt_server/builds/636368209

mysql --version:

mysql  Ver 15.1 Distrib 10.1.41-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

pip freeze: (Currently at Django 2.0, but same error on 2.1 and 2.2)

alabaster==0.7.12
Babel==2.8.0
bootstrap-admin==0.4.3
certifi==2019.11.28
chardet==3.0.4
coverage==4.5.1
decorator==4.1.2
Django==2.0
django-crispy-forms==1.8.1
django-debug-toolbar==1.8
django-nose==1.4.6
django-ratelimit==1.1.0
docutils==0.16
idna==2.8
imagesize==1.2.0
Jinja2==2.10.3
MarkupSafe==1.1.1
mysqlclient==1.3.14
networkx==2.0
nose==1.3.7
packaging==20.0
Pygments==2.5.2
pyparsing==2.4.6
PyPDF2==1.26.0
python-dateutil==2.6.1
pytz==2019.3
requests==2.22.0
six==1.11.0
snowballstemmer==2.0.0
Sphinx==2.3.1
sphinx-rtd-theme==0.4.3
sphinxcontrib-applehelp==1.0.1
sphinxcontrib-devhelp==1.0.1
sphinxcontrib-htmlhelp==1.0.2
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.2
sphinxcontrib-serializinghtml==1.1.3
sqlparse==0.2.3
urllib3==1.25.7

Database settings:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'puzzlehunt_db',
        'HOST': 'localhost',
        'PORT': '3306',
        'USER': 'hunt',
        'PASSWORD': 'NotMyRealMysqlPassword',
        'OPTIONS': {'charset': 'utf8mb4'},
    }
}

I've looked into each of the listed errors (the Transaction issue/the "OperationalError"). A lot of what I found searching keywords wasn't applicable, though I tried many of the solutions anyway to no avail.

If it is relevant, the source code for my whole project can be found here: https://github.com/dlareau/puzzlehunt_server/tree/development

Let me know if there is any other info I can provide.

Thanks for any help!

dlareau commented 4 years ago

I've managed to shrink down the problem causing code quite a bit, still no idea whats happening.

The project is now super simple, its basically just the django-admin startproject/startapp with the urls, templates, tests and models inserted:

project1/
├── app1
│   ├── fixtures
│   │   └── basic.json
│   ├── __init__.py
│   ├── migrations
│   │   ├── 0001_initial.py
│   │   ├── __init__.py
│   ├── models.py
│   ├── templates
│   │   └── index.html
│   ├── tests.py
│   ├── urls.py
│   └── views.py
├── manage.py
├── project1
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
└── requirements.txt

Below are the whole contents of all relevant files. app1/fixtures/basic.json

[
{
    "fields": {
        "name": "Example"
    },
    "model": "app1.model1",
    "pk": 1
}
]

app1/migrations/0001_initial.py

from django.db import migrations, models

class Migration(migrations.Migration):
    initial = True
    dependencies = []
    operations = [
        migrations.CreateModel(
            name='Model1',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('name', models.CharField(max_length=200)),
            ],
        ),
    ]

app1/models.py

from django.db import models

class Model1(models.Model):
    name = models.CharField(max_length=200)

app1/templates/index.html

<html>
</html>

app1/tests.py

from django.test import TestCase
from django.urls import reverse

class InfoTests(TestCase):
    fixtures = ["basic_hunt"]

    def test_index(self):
        response = self.client.get("/")

app1/urls.py

from django.urls import path
from . import views

app_name = "app1"

urlpatterns = [
    path('', views.index, name='index'),
]

app1/views.py

from django.shortcuts import render
from .models import Model1

def index(request):
    curr_model = Model1.objects.get(pk=1)
    return render(request, "index.html", {'curr_model': curr_model})

project1/urls.py

from django.urls import include, path

urlpatterns = [
    path('', include('app1.urls')),
]

project1/settings.py

from os.path import dirname, abspath
import codecs
codecs.register(lambda name: codecs.lookup('utf8') if name == 'utf8mb4' else None)

BASE_DIR = dirname(dirname(dirname(abspath(__file__))))

# Application definition

INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'app1',
    'django_nose',
)

TEST_RUNNER = 'django_nose.NoseTestSuiteRunner'

MIDDLEWARE = (
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'django.middleware.security.SecurityMiddleware',
)

ROOT_URLCONF = 'project1.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.template.context_processors.static',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
                'django.template.context_processors.media',
            ],
        },
    },
]

LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'America/New_York'
USE_I18N = True
USE_L10N = True
USE_TZ = True

DEBUG = True
SECRET_KEY = 'q#)ASes1tP4CAOGnn0oo6N+xM%ZgT2lf1ZVTp2QO)xkF4Jv&*r'
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'puzzlehunt_db',
        'HOST': 'localhost',
        'PORT': '3306',
        'USER': 'hunt',
        'PASSWORD': 'test',
        'OPTIONS': {'charset': 'utf8mb4'},
    }
}
INTERNAL_IPS = '127.0.0.1'

requirements.txt

Django==2.0
mysqlclient==1.4.6
django-nose==1.4.5

Thats literally every line of code in this project and it is still failing on Django 2.0, 2.1 and 2.2, with the same error as above, and still passes when removing the TEST_RUNNER line from settings.py

litrop commented 4 years ago

Because of testcases change in django 2.0, commit is True and connection.close() is run, but I don't know why connection.close() cause the problem.

django-nose 1.4.6 runner.py

def _foreign_key_ignoring_handle(self, *fixture_labels, **options):
    """Wrap the the stock loaddata to ignore foreign key checks.
    This allows loading circular references from fixtures, and is
    monkeypatched into place in setup_databases().
    """
    using = options.get('database', DEFAULT_DB_ALIAS)
    commit = options.get('commit', True)
    connection = connections[using]

    # MySQL stinks at loading circular references:
    if uses_mysql(connection):
        cursor = connection.cursor()
        cursor.execute('SET foreign_key_checks = 0')

    _old_handle(self, *fixture_labels, **options)

    if uses_mysql(connection):
        cursor = connection.cursor()
        cursor.execute('SET foreign_key_checks = 1')

        if commit:
            connection.close()

django 1.11 testcases

call_command('loaddata', *cls.fixtures, **{
    'verbosity': 0,
    'commit': False,
    'database': db_name,
})

django 2.0 testcases

call_command('loaddata', *cls.fixtures, **{'verbosity': 0, 'database': db_name})
dlareau commented 4 years ago

Thanks for tracking that down @Litrop! After a bit of looking I think I have most of it.

It would seem this just needs a code update on django-nose's side. The _foreign_key_ignoring_handle method was meant to take the place of the handle function for the loaddata command. The problem is that loaddata hasn't accepted the "commit" argument since Django 1.5.

Django 2.0 just happened to include a code change that removed the passing of old unused command flags to call_command. This meant that the TestCase's normal commit=False got turned into django-nose's options.get('commit', True), breaking everything.

As for why connection.close() breaks things, some of the details go over my head, but it appears that the original if commit: connection.close line from django 1.5 had the following logic next to it:

# Close the DB connection -- unless we're still in a transaction. This
# is required as a workaround for an edge case in MySQL: if the same
# connection is used to create tables, load data, and query, the query
# can return incorrect results. See Django #7572, MySQL #37735.

Recent versions of django use the following logic instead:

        if transaction.get_autocommit(self.using):
            connections[self.using].close()

It would seem that django-nose should either update to using the same logic, or just let the call to _old_handle deal with it as I think it probably would.

dlareau commented 4 years ago

Another possible note, I don't know enough about MySQL stuff to say, but is it possible that the whole _foreign_key_ignoring_handle method might now not be needed? Maybe either MySQL or Django's loaddata.handle function have improved to the point of making the patching unnecessary.

dlareau commented 4 years ago

Pull request #308 made, all functional tests appear to pass.

zefciu commented 4 years ago

When will we see this change on pypi?

dlareau commented 4 years ago

@zefciu This project appears to be dead, there haven't been any commits in months and my pull request was never merged. I gave up and stopped using it in my personal projects.

al-the-x commented 4 years ago

Hey, @jwhitlock, is this band still touring? #308 could use a little love...

jwhitlock commented 4 years ago

Thanks @dlareau for the bug, analysis, and fix! I've filed #314 for the follow-on issue of 1) testing fixture loading in this project, and 2) evaluating if the MySQL fixture loading workarounds are still needed.

I've merged #308, it ~will be~ is in release 1.4.7.