search

nedbat / django_coverage_plugin

A plugin for coverage.py to measure Django template execution
Apache License 2.0
197 stars 35 forks source link

Requests to external page trrigers an error? #23

Closed oz123 closed 7 years ago

oz123 commented 8 years ago

Some how calling a URL to an external resource inside my view triggers the following exception:

$ DJANGO_SETTINGS_MODULE=sharescreening.settings.devlocal coverage run manage.py test --failfast -v 2
Creating test database for alias 'default' ('/home/oznt/Software/screenshares/sharescreening/../sharescreening/test-db.sqlite3')...
Destroying old test database 'default'...
Operations to perform:
  Synchronize unmigrated apps: staticfiles, messages, select_multiple_field
  Apply all migrations: admin, contenttypes, sharescreening, auth, sessions
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  ... snipped ...
  Applying sharescreening.0023_searchfilterpreview_lang... OK
test_1_create_bookmarks (sharescreening.tests.TestBookmarks) ... ERROR

======================================================================
ERROR: test_1_create_bookmarks (sharescreening.tests.TestBookmarks)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "sharescreening/tests.py", line 282, in setUp
    self._login('normaluser', 'secretz').follow()
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webtest/response.py", line 102, in follow
    return self._follow(**kw)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webtest/response.py", line 90, in _follow
    return self.test_app.get(abslocation, **kw)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django_webtest/__init__.py", line 111, in get
    url, params, headers, extra_environ, status, expect_errors, **kwargs)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webtest/app.py", line 323, in get
    expect_errors=expect_errors)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django_webtest/__init__.py", line 74, in do_request
    response = super(DjangoTestApp, self).do_request(req, status, expect_errors)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webtest/app.py", line 606, in do_request
    res = req.get_response(app, catch_exc_info=True)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webob/request.py", line 1295, in send
    application, catch_exc_info=True)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
    app_iter = application(self.environ, start_response)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webtest/lint.py", line 198, in lint_app
    iterator = application(environ, start_response_wrapper)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py", line 63, in __call__
    return self.application(environ, start_response)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 189, in __call__
    response = self.get_response(request)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 218, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 164, in get_response
    response = response.render()
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/response.py", line 158, in render
    self.content = self.rendered_content
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/response.py", line 135, in rendered_content
    content = template.render(context, self._request)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/backends/django.py", line 74, in render
    return self.template.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/base.py", line 210, in render
    return self._render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/test/utils.py", line 96, in instrumented_test_render
    return self.nodelist.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/base.py", line 905, in render
    bit = self.render_node(node, context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/loader_tags.py", line 135, in render
    return compiled_parent._render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/test/utils.py", line 96, in instrumented_test_render
    return self.nodelist.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/base.py", line 905, in render
    bit = self.render_node(node, context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/loader_tags.py", line 65, in render
    result = block.nodelist.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/base.py", line 905, in render
    bit = self.render_node(node, context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/defaulttags.py", line 161, in render
    values = list(values)
  File "sharescreening/views_utils.py", line 22, in _replace_broken_link
    if check_broken_link(document.get('picture', '')):
  File "sharescreening/utils.py", line 58, in check_broken_link
    h.request('HEAD', parsed.path)
  File "/usr/lib/python2.7/httplib.py", line 979, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1013, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 975, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 835, in _send_output
    self.send(msg)
  File "/usr/lib/python2.7/httplib.py", line 801, in send
    if self.debuglevel > 0:
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django_coverage_plugin/plugin.py", line 170, in line_number_range
    assert frame.f_code.co_name == 'render'
AssertionError

----------------------------------------------------------------------
Ran 1 test in 0.308s

FAILED (errors=1)
Destroying test database for alias 'default' ('/home/oznt/Software/screenshares/sharescreening/../sharescreening/test-db.sqlite3')...

This does not happened with simple testing:

$ DJANGO_SETTINGS_MODULE=sharescreening.settings.devlocal manage.py test --failfast -v 2
Creating test database for alias 'default' ('/home/oznt/Software/screenshares/sharescreening/../sharescreening/test-db.sqlite3')...
Destroying old test database 'default'...
Operations to perform:
  Synchronize unmigrated apps: staticfiles, messages, select_multiple_field
  Apply all migrations: admin, contenttypes, sharescreening, auth, sessions
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying contenttypes.0001_initial... OK
... snipped ...
  Applying sharescreening.0019_auto_20160415_1303... OK
  Applying sharescreening.0020_auto_20160419_1525... OK
  Applying sharescreening.0021_auto_20160419_1600... OK
  Applying sharescreening.0022_auto_20160420_1425... OK
  Applying sharescreening.0023_searchfilterpreview_lang... OK
test_1_create_bookmarks (sharescreening.tests.TestBookmarks) ... ok
test_2_bookmark_list (sharescreening.tests.TestBookmarks) ... ok

Can you tell what is going on here?

nedbat commented 8 years ago

Can you provide instructions so that I can reproduce this error? Does it happen reliably? There's another report of the same assertion, but it is intermittent.

oz123 commented 8 years ago

Hi Ned,

Thanks for the quick reply. Try including the following function in one of your views, and give it a URL:

def check_broken_link(link):
    """:return: True if link is broken"""
    if not link:
        pass

    else:
        url = img_url(link)
        parsed = urlparse.urlparse(url)
        h = httplib.HTTPConnection(parsed.netloc)
        h.request('HEAD', parsed.path)
        response = h.getresponse()
        if response.status == 200:
            return False
    return True
nedbat commented 8 years ago

I haven't tried your suggestion yet. But if you have a complete project I could try, with tests, configuration, etc, that would be much better.

oz123 commented 8 years ago

Unfortunately, it's a commercial non open source code. I could try and reap off the essential parts and make an example project. However, I might be able to get to it next week only.

oz123 commented 8 years ago

Hi Ned, OK, I can tell you definitely that the error happens every time I run the test suite with the django-template-coverage. I also tried creating a new simple project with similar templates and url structure. But I could not recreate the error on a simpler project with the above. Also, if I changed the code of the above posted function to:

import requests
def check_broken_link(link):
    """:return: True if link is broken"""
    try:
        if not link:
            pass

        else:
            url = img_url(link)
            parsed = urlparse.urlparse(url)
            res = requests.get(url)
            if res.status_code == 200:
                return False
        return True
    except AssertionError as E:
        print url
        return True

Now, I am seeing another error:

$ DJANGO_SETTINGS_MODULE=sharescreening.settings.devlocal coverage run manage.py test --failfast -v 2
Creating test database for alias 'default' ('/home/oznt/Software/screenshares/sharescreening/../sharescreening/test-db.sqlite3')...
Destroying old test database 'default'...
Operations to perform:
  Synchronize unmigrated apps: staticfiles, messages, select_multiple_field
  Apply all migrations: admin, contenttypes, sharescreening, auth, sessions
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying sessions.0001_initial... OK
  Applying sharescreening.0001_initial... OK
  Applying sharescreening.0002_auto_20160314_1145... OK
  Applying sharescreening.0003_auto_20160323_1151... OK
  Applying sharescreening.0004_contentbookmark... OK
  Applying sharescreening.0005_auto_20160331_1207... OK
  Applying sharescreening.0006_auto_20160331_1237... OK
...
  Applying sharescreening.0021_auto_20160419_1600... OK
  Applying sharescreening.0022_auto_20160420_1425... OK
  Applying sharescreening.0023_searchfilterpreview_lang... OK
test_1_create_bookmarks (sharescreening.tests.TestBookmarks) ... http://....url ...45947/46.jpg
ERROR

======================================================================
ERROR: test_1_create_bookmarks (sharescreening.tests.TestBookmarks)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/oznt/Software/screenshares/sharescreening/tests.py", line 317, in test_1_create_bookmarks
    response = form.submit().follow()
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webtest/response.py", line 102, in follow
    return self._follow(**kw)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webtest/response.py", line 90, in _follow
    return self.test_app.get(abslocation, **kw)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django_webtest/__init__.py", line 111, in get
    url, params, headers, extra_environ, status, expect_errors, **kwargs)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webtest/app.py", line 323, in get
    expect_errors=expect_errors)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django_webtest/__init__.py", line 74, in do_request
    response = super(DjangoTestApp, self).do_request(req, status, expect_errors)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webtest/app.py", line 606, in do_request
    res = req.get_response(app, catch_exc_info=True)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webob/request.py", line 1295, in send
    application, catch_exc_info=True)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
    app_iter = application(self.environ, start_response)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/webtest/lint.py", line 198, in lint_app
    iterator = application(environ, start_response_wrapper)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py", line 63, in __call__
    return self.application(environ, start_response)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 189, in __call__
    response = self.get_response(request)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 218, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 164, in get_response
    response = response.render()
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/response.py", line 158, in render
    self.content = self.rendered_content
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/response.py", line 135, in rendered_content
    content = template.render(context, self._request)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/backends/django.py", line 74, in render
    return self.template.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/base.py", line 210, in render
    return self._render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/test/utils.py", line 96, in instrumented_test_render
    return self.nodelist.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/base.py", line 905, in render
    bit = self.render_node(node, context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/loader_tags.py", line 135, in render
    return compiled_parent._render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/test/utils.py", line 96, in instrumented_test_render
    return self.nodelist.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/base.py", line 905, in render
    bit = self.render_node(node, context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/loader_tags.py", line 65, in render
    result = block.nodelist.render(context)
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django/template/base.py", line 908, in render
    bits.append(force_text(bit))
  File "/home/oznt/.virtualenvs/screenshares/local/lib/python2.7/site-packages/django_coverage_plugin/plugin.py", line 106, in line_number_range
    source = render_self.source
AttributeError: 'DebugNodeList' object has no attribute 'source'

----------------------------------------------------------------------
Ran 1 test in 0.938s

FAILED (errors=1)
Destroying test database for alias 'default' ('/home/oznt/Software/screenshares/sharescreening/../sharescreening/test-db.sqlite3')...

If I change requests.get to requests.head I am getting the old error.

PamelaM commented 7 years ago

This appears to be a duplicate of https://github.com/nedbat/django_coverage_plugin/issues/22