olivierdalang / django-toosimple-q

Simplistic task queue and cron-like scheduler for Django
MIT License
38 stars 6 forks source link

Error when use with uWSGI #31

Closed paulocoutinhox closed 3 months ago

paulocoutinhox commented 3 months ago

Hi,

Always that i try use it with uWSGI i get error:

uwsgi --ini uwsgi.ini
[uWSGI] getting INI configuration from uwsgi.ini
[uwsgi-static] added mapping for /static => /app/static
[uwsgi-static] added mapping for /media => /app/media
*** Starting uWSGI 2.0.26 (64bit) on [Tue Jul  2 20:40:26 2024] ***
compiled with version: 12.2.0 on 02 July 2024 20:22:17
os: Linux-6.6.16-linuxkit #1 SMP Fri Feb 16 11:54:02 UTC 2024
nodename: 93cb28ccef8f
machine: aarch64
clock source: unix
pcre jit disabled
detected number of CPU cores: 10
current working directory: /app
detected binary path: /usr/local/bin/uwsgi
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** 
chdir() to /app
limiting address space of processes...
your process address space limit is 268435456 bytes (256 MB)
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
building mime-types dictionary from file /etc/mime.types...1545 entry found
lock engine: pthread robust mutexes
thunder lock: enabled
uwsgi socket 0 bound to UNIX address /tmp/pyaa.sock fd 3
uwsgi socket 1 bound to TCP address :8000 fd 4
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** 
Python version: 3.10.14 (main, Jun 27 2024, 03:48:43) [GCC 12.2.0]
Python main interpreter initialized at 0xaaab1067c070
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** 
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 2449018 bytes (2391 KB) for 20 cores
*** Operational MODE: preforking+threaded ***
[2024-07-02 20:40:27][INFO][toosimpleq] Autodiscovering tasks.py...
OpenBLAS blas_thread_init: pthread_create failed for thread 8 of 10: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
OpenBLAS blas_thread_init: pthread_create failed for thread 9 of 10: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max
Traceback (most recent call last):
  File "/app/./pyaa/wsgi.py", line 16, in <module>
    application = get_wsgi_application()
  File "/usr/local/lib/python3.10/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
    django.setup(set_prefix=False)
  File "/usr/local/lib/python3.10/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python3.10/site-packages/django/apps/registry.py", line 124, in populate
    app_config.ready()
  File "/usr/local/lib/python3.10/site-packages/django_toosimple_q/apps.py", line 15, in ready
    autodiscover_modules("tasks")
  File "/usr/local/lib/python3.10/site-packages/django/utils/module_loading.py", line 58, in autodiscover_modules
    import_module("%s.%s" % (app_config.name, module_to_search))
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/app/./apps/task_queue/tasks.py", line 5, in <module>
    import cv2
  File "/usr/local/lib/python3.10/site-packages/cv2/__init__.py", line 11, in <module>
    import numpy
  File "/usr/local/lib/python3.10/site-packages/numpy/__init__.py", line 130, in <module>
    from numpy.__config__ import show as show_config
  File "/usr/local/lib/python3.10/site-packages/numpy/__config__.py", line 4, in <module>
    from numpy.core._multiarray_umath import (
  File "/usr/local/lib/python3.10/site-packages/numpy/core/__init__.py", line 24, in <module>
    from . import multiarray
  File "/usr/local/lib/python3.10/site-packages/numpy/core/multiarray.py", line 10, in <module>
    from . import overrides
  File "/usr/local/lib/python3.10/site-packages/numpy/core/overrides.py", line 8, in <module>
    from numpy.core._multiarray_umath import (
  File "<frozen importlib._bootstrap>", line 216, in _lock_unlock_module
KeyboardInterrupt
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** 
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 7)

It dont happen when i run:

python3 manage.py runserver

or

python3 manager.py worker

Can you help me?

Thanks.

olivierdalang commented 3 months ago

Hey ! Thanks for opening an issue.

From your trace, it looks like this is happening when toosimple_q autoloads the tasks.py modules from your apps. At this stage, not much else beyond importing those modules happens.

Can you reproduce your issue without toosimple_q, just by explicitly importing the tasks.py modules at startup ?

paulocoutinhox commented 3 months ago

I abandon this way and pass the iA part to other container isolated