lemonsaurus / django-simple-bulma

Django application to add the Bulma CSS framework and its extensions
MIT License
136 stars 16 forks source link

Intermittent "circular import" when starting Django app #71

Closed perlun closed 3 years ago

perlun commented 3 years ago

Hi,

Thanks for a nice Python/Django package. It works fine, but sometimes I get this error on startup with this Django app installed (as described in the installation details):

pipenv run ./manage.py runserver
Watching for file changes with StatReloader
2021-04-20 13:37:09,008 INFO [django.utils.autoreload] Watching for file changes with StatReloader
Performing system checks...

Traceback (most recent call last):
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 66, in __getitem__
    return self._engines[alias]
KeyError: 'django'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 121, in get_package_libraries
    module = import_module(entry[1])
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django_simple_bulma/templatetags/django_simple_bulma.py", line 11, in <module>
    from ..utils import (
ImportError: cannot import name 'fontawesome_token' from partially initialized module 'django_simple_bulma.utils' (most likely due to a circular import) (/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django_simple_bulma/utils.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/per/git/centraln/./manage.py", line 22, in <module>
    main()
  File "/home/per/git/centraln/./manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 61, in execute
    super().execute(*args, **options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 96, in handle
    self.run(**options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 103, in run
    autoreload.run_with_reloader(self.inner_run, **options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 637, in run_with_reloader
    start_django(reloader, main_func, *args, **kwargs)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 622, in start_django
    reloader.run(django_main_thread)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 327, in run
    autoreload_started.send(sender=self)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/dispatch/dispatcher.py", line 180, in send
    return [
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/dispatch/dispatcher.py", line 181, in <listcomp>
    (receiver, receiver(signal=self, sender=sender, **named))
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/autoreload.py", line 41, in watch_for_template_changes
    for directory in get_template_directories():
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/autoreload.py", line 14, in get_template_directories
    for backend in engines.all():
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 90, in all
    return [self[alias] for alias in self]
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 90, in <listcomp>
    return [self[alias] for alias in self]
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 81, in __getitem__
    engine = engine_cls(params)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 25, in __init__
    options['libraries'] = self.get_templatetag_libraries(libraries)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 43, in get_templatetag_libraries
    libraries = get_installed_libraries()
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 108, in get_installed_libraries
    for name in get_package_libraries(pkg):
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 123, in get_package_libraries
    raise InvalidTemplateLibrary(
django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'django_simple_bulma.templatetags.django_simple_bulma': cannot import name 'fontawesome_token' from partially initialized module 'django_simple_bulma.utils' (most likely due to a circular import) (/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django_simple_bulma/utils.py)
make: *** [Makefile:24: serve] Error 1

Note that this only happens sometimes, which is both puzzling and annoying. It can also happen when making a change to any .py file in the project, i.e. when the apps are reloaded. Any ideas? Thanks in advance.

perlun commented 3 years ago

One idea I had was that the load_django_simple_bulma from our base.html HTML template:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    {% load static %}
    {% load django_simple_bulma %}
    {% bulma %}
    <link rel="shortcut icon" href="{% static 'favicon.ico' %}"/>
    <title>{% block title %}{% endblock %}</title>
</head>
<body>

...would trigger the loading of the django_simple_bulma app. But, in our settings.py file, this app was actually listed after our "main" apps (index and hibox_db). So, my theory was that it would basically create a race condition between "starting up the index app" and "starting up the django_simple_bulma" app.

I thus tried rearranging my apps like this:

diff --git centraln/settings.py centraln/settings.py
index d94ee61..710e438 100644
--- centraln/settings.py
+++ centraln/settings.py
@@ -33,8 +33,6 @@ ALLOWED_HOSTS = ['127.0.0.1']
 # Application definition

 INSTALLED_APPS = [
-    'index',
-    'hibox_db',
     'simple_history',
     'django_simple_bulma',
     'django.contrib.admin',
@@ -43,6 +41,9 @@ INSTALLED_APPS = [
     'django.contrib.sessions',
     'django.contrib.messages',
     'django.contrib.staticfiles',
+
+    'index',
+    'hibox_db',
 ]

 MIDDLEWARE = [

...but unfortunately it still didn't work. I'm still seeing this problem intermittently. :cry:

For reference, our django-simple-bulma version is 2.2.0.

lemonsaurus commented 3 years ago

I'm not sure exactly why this is happening, but I suspect something to do with INSTALLED_APPS load order.

Here's the load order we've been using on pages like https://pythondiscord.org for years now without incident:

INSTALLED_APPS = [
    'index',  # put all your own local apps at the start here.

    'django.contrib.admin',  # all the built-in django stuff here.

    'django-simple-bulma',  # Third party django apps go in the end.
]

Can you try this and report back if it still happens?

perlun commented 3 years ago

Thanks. :+1: :bow: I tried this now and re-ran pipenv run ./manage.py runserver about 5-6 times. No errors thus far. I'll keep you in the loop if I happen to see this again.

(The important part of the change: moving django_simple_bulma after django.contrib.admin in INSTALLED_APPS, that might have done the trick actually.)

lemonsaurus commented 3 years ago

Great to hear! I'll close this issue for now then, but feel free to re-open it if you have any more issues!

perlun commented 3 years ago

Tack så mycket, jag gör så. 😊👍🏻

perlun commented 3 years ago

Hmm, maybe we jumped to conclusions too early. Still seeing this sometimes, at least on reloads:

/home/per/git/centraln/index/migrations/0001_initial.py changed, reloading.
2021-04-28 14:14:09,267 INFO [django.utils.autoreload] /home/per/git/centraln/index/migrations/0001_initial.py changed, reloading.
Watching for file changes with StatReloader
2021-04-28 14:14:09,597 INFO [django.utils.autoreload] Watching for file changes with StatReloader
Performing system checks...

Traceback (most recent call last):
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 66, in __getitem__
    return self._engines[alias]
KeyError: 'django'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 121, in get_package_libraries
    module = import_module(entry[1])
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django_simple_bulma/templatetags/django_simple_bulma.py", line 11, in <module>
    from ..utils import (
ImportError: cannot import name 'fontawesome_token' from partially initialized module 'django_simple_bulma.utils' (most likely due to a circular import) (/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django_simple_bulma/utils.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/per/git/centraln/./manage.py", line 22, in <module>
    main()
  File "/home/per/git/centraln/./manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 61, in execute
    super().execute(*args, **options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 96, in handle
    self.run(**options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 103, in run
    autoreload.run_with_reloader(self.inner_run, **options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 637, in run_with_reloader
    start_django(reloader, main_func, *args, **kwargs)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 622, in start_django
    reloader.run(django_main_thread)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 327, in run
    autoreload_started.send(sender=self)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/dispatch/dispatcher.py", line 180, in send
    return [
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/dispatch/dispatcher.py", line 181, in <listcomp>
    (receiver, receiver(signal=self, sender=sender, **named))
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/autoreload.py", line 41, in watch_for_template_changes
    for directory in get_template_directories():
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/autoreload.py", line 14, in get_template_directories
    for backend in engines.all():
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 90, in all
    return [self[alias] for alias in self]
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 90, in <listcomp>
    return [self[alias] for alias in self]
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 81, in __getitem__
    engine = engine_cls(params)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 25, in __init__
    options['libraries'] = self.get_templatetag_libraries(libraries)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 43, in get_templatetag_libraries
    libraries = get_installed_libraries()
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 108, in get_installed_libraries
    for name in get_package_libraries(pkg):
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 123, in get_package_libraries
    raise InvalidTemplateLibrary(
django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'django_simple_bulma.templatetags.django_simple_bulma': cannot import name 'fontawesome_token' from partially initialized module 'django_simple_bulma.utils' (most likely due to a circular import) (/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django_simple_bulma/utils.py)
make: *** [Makefile:26: serve] Error 1
lemonsaurus commented 3 years ago

Hopefully resolved by #72, which was released with version 2.3.1.

perlun commented 3 years ago

Haven't upgraded, but using the locally patched version gave me an error like this today. Honestly don't know if django-simple-bulma was involved in this at all. :grin: It happened as the app was reloaded when I saved some changes in my IDE (IntelliJ Ultimate):

Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 64, in wrapper
    fn(*args, **kwargs)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 118, in inner_run
    self.check(display_num_errors=True)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/base.py", line 419, in check
    all_issues = checks.run_checks(
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/checks/registry.py", line 76, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/checks/urls.py", line 13, in check_url_config
    return check_resolver(resolver)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/checks/urls.py", line 23, in check_resolver
    return check_method()
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/urls/resolvers.py", line 412, in check
    for pattern in self.url_patterns:
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/urls/resolvers.py", line 598, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/urls/resolvers.py", line 591, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/per/git/centraln/centraln/urls.py", line 23, in <module>
    path('', include('index.urls'), name="index"),
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/urls/conf.py", line 34, in include
    urlconf_module = import_module(urlconf_module)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/per/git/centraln/index/urls.py", line 4, in <module>
    from .customer import views as customer_views
  File "/home/per/git/centraln/index/customer/views.py", line 4, in <module>
    from index.filters import AutoSubmittingChoiceFilter, AutoSubmittingBooleanFilter, AutoSubmittingModelChoiceFilter
  File "/home/per/git/centraln/index/filters.py", line 5, in <module>
    from index.server.views import NameField
  File "/home/per/git/centraln/index/server/views.py", line 7, in <module>
    from index.filters import AutoSubmittingChoiceFilter, AutoSubmittingBooleanFilter
ImportError: cannot import name 'AutoSubmittingChoiceFilter' from partially initialized module 'index.filters' (most likely due to a circular import) (/home/per/git/centraln/index/filters.py)
perlun commented 3 years ago

Heh. Please disregard that one. It was a genuine circular dependency in my own app... :see_no_evil: :see_no_evil: :see_no_evil: :grin: I found and fixed it now.

lemonsaurus commented 3 years ago

Haha, no problem. I appreciate your willingness to let us know!