macropy3 1.0.4.dev2 is in conflict with django's importlib

[chfw]

Hi

javascripthon has brought us here. Our django users are suffering from conflicting import loaders. Could you please help have a look?

Problem

(test-env)  jaska$ python manage.py runserver
Performing system checks...

System check identified no issues (0 silenced).
Unhandled exception in thread started by <function check_errors.<locals>.wrapper at 0x10476c7b8>
Traceback (most recent call last):
  File "/Users/jaska/github/pyecharts/pyecharts-users-cases/demo-django/myechartsite/test-env/lib/python3.6/site-packages/django/utils/six.py", line 189, in __get_module
    return self.known_modules[fullname]
KeyError: 'django.contrib.messages.migrations'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jaska/github/pyecharts/pyecharts-users-cases/demo-django/myechartsite/test-env/lib/python3.6/site-packages/django/utils/autoreload.py", line 225, in wrapper
    fn(*args, **kwargs)
  File "/Users/jaska/github/pyecharts/pyecharts-users-cases/demo-django/myechartsite/test-env/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 123, in inner_run
    self.check_migrations()
  File "/Users/jaska/github/pyecharts/pyecharts-users-cases/demo-django/myechartsite/test-env/lib/python3.6/site-packages/django/core/management/base.py", line 427, in check_migrations
    executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
  File "/Users/jaska/github/pyecharts/pyecharts-users-cases/demo-django/myechartsite/test-env/lib/python3.6/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/Users/jaska/github/pyecharts/pyecharts-users-cases/demo-django/myechartsite/test-env/lib/python3.6/site-packages/django/db/migrations/loader.py", line 49, in __init__
    self.build_graph()
  File "/Users/jaska/github/pyecharts/pyecharts-users-cases/demo-django/myechartsite/test-env/lib/python3.6/site-packages/django/db/migrations/loader.py", line 200, in build_graph
    self.load_disk()
  File "/Users/jaska/github/pyecharts/pyecharts-users-cases/demo-django/myechartsite/test-env/lib/python3.6/site-packages/django/db/migrations/loader.py", line 77, in load_disk
    module = import_module(module_name)
  File "/Users/jaska/github/pyecharts/pyecharts-users-cases/demo-django/myechartsite/test-env/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 951, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 894, in _find_spec
  File "/Users/jaska/github/pyecharts/pyecharts-users-cases/demo-django/myechartsite/test-env/lib/python3.6/site-packages/macropy/core/import_hooks.py", line 134, in find_spec
    source = spec.loader.get_source(fullname)
  File "/Users/jaska/github/pyecharts/pyecharts-users-cases/demo-django/myechartsite/test-env/lib/python3.6/site-packages/django/utils/six.py", line 220, in get_code
    self.__get_module(fullname)  # eventually raises ImportError
  File "/Users/jaska/github/pyecharts/pyecharts-users-cases/demo-django/myechartsite/test-env/lib/python3.6/site-packages/django/utils/six.py", line 191, in __get_module
    raise ImportError("This loader does not know module " + fullname)
ImportError: This loader does not know module django.contrib.messages.migrations

reproducing steps:

git clone https://github.com/chfw/pyecharts-users-cases.git
cd pyecharts-users-cases/demo-django/myechartsite

virtualenv --no-site-packages your-env  # please use python 3.6
source your-env/bin/activate

pip install -r requirements.txt
python manage.py runserver

why macropy3?

If the MacroLoader is commented out, the problem is gone. is Django import system wrong, from your point of view?

"""This is the root directory of the project, and directly contains a bunch of
stable, useful and well-tested macros. It also sets up the import hooks that are
required for macros to run properly"""

def activate():
    from .core import macros
    from .core import cleanup
    from .core import exact_src
    from .core import gen_sym

    from .core import import_hooks
    import sys
    #sys.meta_path.insert(0, import_hooks.MacroFinder)
    import macropy
    from .core import hquotes
    from .core import failure

def console():
    from macropy.core.console import MacroConsole
    MacroConsole().interact("0=[]=====> MacroPy Enabled <=====[]=0")

from .core import exporters

__version__ = "1.0.4.dev2"
exporter = exporters.NullExporter()

references:

https://github.com/pyecharts/pyecharts/issues/541 https://github.com/kinegratii/django-echarts/issues/17

[azazel75]

That's a tough a one, thanks for reporting it

[azazel75]

@chfw, I've tried your steps to reproduce the error, but django loads up without issues, I'm running it on linux... is there anything else I should do to see the actual error?

[azazel75]

I have the suspect that the error you get is caused by the platform you're running it on (i.e. Windows) but I haven't the means to confirm that. Anyway it's an error for MacroPy3's import hooks to raise exceptions so I've shielded them against exceptions raised by code called by them. I've consequentely released MacroPy v1.1.b2 and JavaScripthon 0.10. @chfw can you try please using the latter one in your package and tell me if this error is fixed?

[chfw]

With the new releases, the exception is gone. Feels good. Thanks for your support!!

This issue was reproducing on Windows and MacOS.