ImproperlyConfigured exception when running celery worker through Kolo

[mxsasha]

As we discussed at DjangoCon, I'm trying to run Celery workers from this project through Kolo.

Command line: python -m kolo run celery -A internetnl worker results in the exception:

Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/kolo/__main__.py", line 155, in <module>
    cli()  # pragma: no cover
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/kolo/__main__.py", line 71, in run
    profile_module(profiler, path)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/kolo/__main__.py", line 21, in profile_module
    run_module(module_name, run_name="__main__", alter_sys=True)
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 210, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/celery/__main__.py", line 19, in <module>
    main()
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/celery/__main__.py", line 15, in main
    sys.exit(_main())
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/celery/bin/celery.py", line 217, in main
    return celery(auto_envvar_prefix="CELERY")
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/celery/bin/base.py", line 134, in caller
    return f(ctx, *args, **kwargs)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/celery/bin/worker.py", line 343, in worker
    worker = app.Worker(
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/celery/worker/worker.py", line 94, in __init__
    self.app.loader.init_worker()
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/celery/loaders/base.py", line 111, in init_worker
    self.import_default_modules()
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/celery/loaders/base.py", line 105, in import_default_modules
    raise response
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/celery/utils/dispatch/signal.py", line 276, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/celery/fixups/django.py", line 82, in on_import_modules
    self.worker_fixup.validate_models()
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/celery/fixups/django.py", line 120, in validate_models
    self.django_setup()
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/celery/fixups/django.py", line 116, in django_setup
    django.setup()
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/django/apps/config.py", line 255, in create
    return app_config_class(app_name, app_module)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/django/apps/config.py", line 49, in __init__
    self.path = self._path_from_module(app_module)
  File "/Users/sasha/dev/internet.nl/.venv/lib/python3.9/site-packages/django/apps/config.py", line 88, in _path_from_module
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: The app module <module 'frontend' (namespace)> has multiple filesystem locations (['/Users/sasha/dev/internet.nl/frontend', '/Users/sasha/dev/internet.nl/./frontend']); you must configure this app with an AppConfig subclass with a 'path' class attribute.

This is Kolo 2.0.2 on Python 3.9.12 on Mac OS.

[LilyFoote]

I've tried looking into this, but I haven't managed to replicate the bug.

I tried to clone and set up the internet.nl project locally, but I couldn't get unbound to compile properly.

I also tried to replicate using a different project that has celery, but I didn't get any error.

[mxsasha]

Yes, the project setup is complex and this is an obstacle. I have some plans to improve our dependency situation, so let's see if we can get back to this once it's easier to run the project. At least it looks like it's not a generic celery problem, perhaps we're doing something obscure. Thanks for looking into it!

[LilyFoote]

Thanks for reporting it. I hope we get to the bottom of the bug sometime. Maybe what Kolo's doing is ok for most people but breaks some more complex setups.