search

hitchtest / hitch

Loosely coupled testing framework
GNU Affero General Public License v3.0
112 stars 9 forks source link

celery problem? #22

Open musicformellons opened 8 years ago

musicformellons commented 8 years ago

I trying the stub.test but get this:

[ Setup Postgres] /home/usr/.hitchpkg/postgresql-9.4.5/postgresql-9.4.5/bin/pg_ctl -D /home/usr/pq/tests/.hitch/pgdata -l logfile start [ Err Setup Postgres] WARNING: enabling "trust" authentication for local connections [ Err Setup Postgres] You can change this by editing pg_hba.conf or using the option -A, or [ Err Setup Postgres] --auth-local and --auth-host, the next time you run initdb. [ Hitch] Starting Postgres [ Hitch] Directory: /home/usr/pq/tests/.hitch [ Hitch] Command: /home/usr/.hitchpkg/postgresql-9.4.5/postgresql-9.4.5/bin/postgres -p 15432 -D /home/usr/pq/tests/.hitch/pgdata --unix_socket_directories=/home/usr/pq/tests/.hitch/pgdata --log_destination=stderr [ Err Postgres] LOG: database system was shut down at 2016-01-18 21:46:01 CET [ Err Postgres] LOG: MultiXact member wraparound protections are now enabled [ Err Postgres] LOG: database system is ready to accept connections [ Hitch] Postgres Loaded. [ Err Postgres] LOG: autovacuum launcher started [ Post Postgres] Creating users and databases... [ Post Postgres] CREATE ROLE [ Post Postgres] CREATE DATABASE [ Setup Django] Running migrations... [ Hitch] Starting Celery [ Hitch] Directory: /home/usr/pq [ Hitch] Command: /home/usr/pq/tests/.hitch/py3.4.3/bin/python -u -m celery worker --app=remindme --loglevel=INFO --concurrency=2 [ Err Celery] /home/usr/pq/tests/.hitch/py3.4.3/bin/python: No module named celery WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv. Python 3.4.3 (default, Sep 15 2015, 12:35:19) Type "copyright", "credits" or "license" for more information.

IPython 4.0.2 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details.

Exception occurred in "Stub" setup /home/usr/pq/tests/stub.test --tags feature-name

/home/usr/pq/tests/engine.py

100 :         )
101 :

--> 102 : self.services.startup(interactive=False) 103 : 104 : # Docs : https://hitchtest.readthedocs.org/en/latest/plugins/hitchcli.html

/home/usr/pq/tests/.hitch/virtualenv/lib/python3.4/site-packages/hitchserve/service_bundle.py

194 :             if interactive:
195 :                 self.start_interactive_mode()

--> 196 : raise what_happened 197 : else: 198 : if interactive:

hitchserve.hitch_exception.ServiceSuddenStopException

Service 'Celery' stopped suddenly.

In 1:

crdoconnor commented 8 years ago

Hi! Thanks for reporting.

It looks like your requirements.txt might not contain celery. Can you show me what's in /home/usr/pq/requirements.txt ?

musicformellons commented 8 years ago

you're right, I added celery to requirements, now I get:

[ Err Setup Postgres] --auth-local and --auth-host, the next time you run initdb. [ Hitch] Starting Postgres [ Hitch] Directory: /home/usr/pq/tests/.hitch [ Hitch] Command: /home/usr/.hitchpkg/postgresql-9.4.5/postgresql-9.4.5/bin/postgres -p 15432 -D /home/usr/pq/tests/.hitch/pgdata --unix_socket_directories=/home/usr/pq/tests/.hitch/pgdata --log_destination=stderr [ Err Postgres] LOG: database system was shut down at 2016-01-19 14:40:33 CET [ Err Postgres] LOG: MultiXact member wraparound protections are now enabled [ Err Postgres] LOG: database system is ready to accept connections [ Hitch] Postgres Loaded. [ Err Postgres] LOG: autovacuum launcher started [ Post Postgres] Creating users and databases... [ Post Postgres] CREATE ROLE [ Post Postgres] CREATE DATABASE [ Setup Django] Running migrations... [ Hitch] Starting Celery [ Hitch] Directory: /home/usr/pq [ Hitch] Command: /home/usr/pq/tests/.hitch/py3.4.3/bin/python -u -m celery worker --app=remindme --loglevel=INFO --concurrency=2 [ Err Setup Django] Traceback (most recent call last): [ Err Setup Django] File "/home/usr/pq/manage.py", line 10, in [ Err Setup Django] execute_from_command_line(sys.argv) [ Err Setup Django] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/django/core/management/init.py", line 351, in execute_from_command_line [ Err Setup Django] utility.execute() [ Err Setup Django] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/django/core/management/init.py", line 303, in execute [ Err Setup Django] settings.INSTALLED_APPS [ Err Setup Django] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/django/conf/init.py", line 48, in getattr [ Err Setup Django] self._setup(name) [ Err Setup Django] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/django/conf/init.py", line 44, in _setup [ Err Setup Django] self._wrapped = Settings(settings_module) [ Err Setup Django] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/django/conf/init.py", line 92, in init [ Err Setup Django] mod = importlib.import_module(self.SETTINGS_MODULE) [ Err Setup Django] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/importlib/init.py", line 109, in import_module [ Err Setup Django] return _bootstrap._gcd_import(name[level:], package, level) [ Err Setup Django] File "", line 2254, in _gcd_import [ Err Setup Django] File "", line 2237, in _find_and_load [ Err Setup Django] File "", line 2212, in _find_and_load_unlocked [ Err Setup Django] File "", line 321, in _call_with_frames_removed [ Err Setup Django] File "", line 2254, in _gcd_import [ Err Setup Django] File "", line 2237, in _find_and_load [ Err Setup Django] File "", line 2224, in _find_and_load_unlocked [ Err Setup Django] ImportError: No module named 'remindme' [ Hitch] Starting Django [ Hitch] Directory: /home/usr/pq [ Hitch] Command: /home/usr/pq/tests/.hitch/py3.4.3/bin/python -u /home/usr/pq/manage.py runserver 18080 --verbosity 1 --noreload --nothreading --settings=remindme.settings Python 3.4.3 (default, Sep 15 2015, 12:35:19) Type "copyright", "credits" or "license" for more information.

IPython 4.0.2 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details.

Exception occurred in "Stub" setup /home/usr/pq/tests/stub.test --tags feature-name

/home/usr/pq/tests/engine.py

100 :         )
101 :

--> 102 : self.services.startup(interactive=False) 103 : 104 : # Docs : https://hitchtest.readthedocs.org/en/latest/plugins/hitchcli.html

/home/usr/pq/tests/.hitch/virtualenv/lib/python3.4/site-packages/hitchserve/service_bundle.py

190 :             if interactive:
191 :                 self.start_interactive_mode()

--> 192 : six.reraise(*what_happened) 193 : elif isinstance(what_happened, Exception): 194 : if interactive:

/home/usr/pq/tests/.hitch/virtualenv/lib/python3.4/site-packages/six.py

683 :             value = tp()
684 :         if value.__traceback__ is not tb:

--> 685 : raise value.with_traceback(tb) 686 : raise value 687 :

/home/usr/pq/tests/.hitch/virtualenv/lib/python3.4/site-packages/hitchserve/service_handle.py

42 :                 sys.stdout = open(self.bundle_engine.service_bundle.hitch_dir.setup_out(self.service.name), "ab", 0)
43 :                 sys.stderr = open(self.bundle_engine.service_bundle.hitch_dir.setup_err(self.service.name), "ab", 0)

--> 44 : self.service.setup() 45 : except Exception as e: 46 : pickling_support.install()

/home/usr/pq/tests/.hitch/virtualenv/lib/python3.4/site-packages/hitchpython/django_service.py

91 :         if self.migrations:
92 :             self.log("Running migrations...")

--> 93 : self.manage("migrate").run() 94 : if self.sites: 95 : self.log("Updating sites table...")

/home/usr/pq/tests/.hitch/virtualenv/lib/python3.4/site-packages/hitchserve/hitch_service.py

76 :             else:
77 :                 kwargs['stdout'] = sys.stdout

--> 78 : return subprocess.check_call(self.command, **kwargs) 79 : except subprocess.CalledProcessError: 80 : if ignore_errors:

/home/usr/.pyenv/versions/3.4.3/lib/python3.4/subprocess.py

559 :         if cmd is None:
560 :             cmd = popenargs[0]

--> 561 : raise CalledProcessError(retcode, cmd) 562 : return 0 563 :

subprocess.CalledProcessError

Command '['/home/usr/pq/tests/.hitch/py3.4.3/bin/python', '-u', '/home/usr/pq/manage.py', 'migrate', '--settings=remindme.settings']' returned non-zero exit status 1

In 1:

crdoconnor commented 8 years ago

Change 'remindme.settings' to the settings file you want to run to run your django app.

That's still a bug I need to fix: https://github.com/hitchtest/hitchquickstart/issues/2

musicformellons commented 8 years ago

Ok, tried to change the remindme.settings to point to my django settings (local.py) file. I suppose my Django app is the name of my django project!? Anyway, I now get:

[ Setup Django] Running migrations... [ Hitch] Starting Celery [ Hitch] Directory: /home/usr/pq [ Hitch] Command: /home/usr/pq/tests/.hitch/py3.4.3/bin/python -u -m celery worker --app=pq --loglevel=INFO --concurrency=2 [ Err Setup Django] Traceback (most recent call last): [ Err Setup Django] File "/home/usr/pq/manage.py", line 10, in [ Err Setup Django] execute_from_command_line(sys.argv) [ Err Setup Django] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/django/core/management/init.py", line 351, in execute_from_command_line [ Err Setup Django] utility.execute() [ Err Setup Django] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/django/core/management/init.py", line 303, in execute [ Err Setup Django] settings.INSTALLED_APPS [ Err Setup Django] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/django/conf/init.py", line 48, in getattr [ Err Setup Django] self._setup(name) [ Err Setup Django] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/django/conf/init.py", line 44, in _setup [ Err Setup Django] self._wrapped = Settings(settings_module) [ Err Setup Django] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/django/conf/init.py", line 92, in init [ Err Setup Django] mod = importlib.import_module(self.SETTINGS_MODULE) [ Err Setup Django] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/importlib/init.py", line 109, in import_module [ Err Setup Django] return _bootstrap._gcd_import(name[level:], package, level) [ Err Setup Django] File "", line 2254, in _gcd_import [ Err Setup Django] File "", line 2237, in _find_and_load [ Err Setup Django] File "", line 2212, in _find_and_load_unlocked [ Err Setup Django] File "", line 321, in _call_with_frames_removed [ Err Setup Django] File "", line 2254, in _gcd_import [ Err Setup Django] File "", line 2237, in _find_and_load [ Err Setup Django] File "", line 2212, in _find_and_load_unlocked [ Err Setup Django] File "", line 321, in _call_with_frames_removed [ Err Setup Django] File "", line 2254, in _gcd_import [ Err Setup Django] File "", line 2237, in _find_and_load [ Err Setup Django] File "", line 2224, in _find_and_load_unlocked [ Err Setup Django] ImportError: No module named 'pq.config' [ Hitch] Starting Django [ Hitch] Directory: /home/usr/pq [ Hitch] Command: /home/usr/pq/tests/.hitch/py3.4.3/bin/python -u /home/usr/pq/manage.py runserver 18080 --verbosity 1 --noreload --nothreading --settings=pq.config.settings.local Python 3.4.3 (default, Sep 15 2015, 12:35:19) Type "copyright", "credits" or "license" for more information.

IPython 4.0.2 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details.

Exception occurred in "Stub" setup /home/usr/pq/tests/stub.test --tags feature-name

/home/usr/pq/tests/engine.py

100 :         )
101 :

--> 102 : self.services.startup(interactive=False) 103 : 104 : # Docs : https://hitchtest.readthedocs.org/en/latest/plugins/hitchcli.html

/home/usr/pq/tests/.hitch/virtualenv/lib/python3.4/site-packages/hitchserve/service_bundle.py

190 :             if interactive:
191 :                 self.start_interactive_mode()

--> 192 : six.reraise(*what_happened) 193 : elif isinstance(what_happened, Exception): 194 : if interactive:

/home/usr/pq/tests/.hitch/virtualenv/lib/python3.4/site-packages/six.py

683 :             value = tp()
684 :         if value.__traceback__ is not tb:

--> 685 : raise value.with_traceback(tb) 686 : raise value 687 :

/home/usr/pq/tests/.hitch/virtualenv/lib/python3.4/site-packages/hitchserve/service_handle.py

42 :                 sys.stdout = open(self.bundle_engine.service_bundle.hitch_dir.setup_out(self.service.name), "ab", 0)
43 :                 sys.stderr = open(self.bundle_engine.service_bundle.hitch_dir.setup_err(self.service.name), "ab", 0)

--> 44 : self.service.setup() 45 : except Exception as e: 46 : pickling_support.install()

/home/usr/pq/tests/.hitch/virtualenv/lib/python3.4/site-packages/hitchpython/django_service.py

91 :         if self.migrations:
92 :             self.log("Running migrations...")

--> 93 : self.manage("migrate").run() 94 : if self.sites: 95 : self.log("Updating sites table...")

/home/usr/pq/tests/.hitch/virtualenv/lib/python3.4/site-packages/hitchserve/hitch_service.py

76 :             else:
77 :                 kwargs['stdout'] = sys.stdout

--> 78 : return subprocess.check_call(self.command, **kwargs) 79 : except subprocess.CalledProcessError: 80 : if ignore_errors:

/home/usr/.pyenv/versions/3.4.3/lib/python3.4/subprocess.py

559 :         if cmd is None:
560 :             cmd = popenargs[0]

--> 561 : raise CalledProcessError(retcode, cmd) 562 : return 0 563 :

subprocess.CalledProcessError

Command '['/home/usr/pq/tests/.hitch/py3.4.3/bin/python', '-u', '/home/usr/pq/manage.py', 'migrate', '--settings=pq.config.settings.local']' returned non-zero exit status 1

In 1:

crdoconnor commented 8 years ago

It still doesn't like your specified settings for some reason.

The critical error message is here::

[ Err Setup Django] ImportError: No module named 'pq.config'

It's trying to run:

/home/usr/pq/tests/.hitch/py3.4.3/bin/python -u /home/usr/pq/manage.py runserver 18080 --verbosity 1 --noreload --nothreading --settings=pq.config.settings.local

I think you probably needed to put in: "config.settings.local" rather than "pq.config.settings.local".

musicformellons commented 8 years ago

mmh, changed it to config.setting.local looks like it gets stuck earlier now: [ Post Postgres] CREATE DATABASE [ Setup Django] Running migrations... [ Hitch] Starting Celery [ Hitch] Directory: /home/usr/pq [ Hitch] Command: /home/usr/pq/tests/.hitch/py3.4.3/bin/python -u -m celery worker --app=pq --loglevel=INFO --concurrency=2 [ Err Celery] Traceback (most recent call last): [ Err Celery] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/celery/app/utils.py", line 235, in find_app [ Err Celery] found = sym.app [ Err Celery] AttributeError: 'module' object has no attribute 'app' [ Err Celery] During handling of the above exception, another exception occurred: [ Err Celery] Traceback (most recent call last): [ Err Celery] File "/home/usr/.hitchpkg/python3.4.3/lib/python3.4/runpy.py", line 170, in _run_module_as_main [ Err Celery] "main", mod_spec) [ Err Celery] File "/home/usr/.hitchpkg/python3.4.3/lib/python3.4/runpy.py", line 85, in _run_code [ Err Celery] exec(code, run_globals) [ Err Celery] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/celery/main.py", line 54, in [ Err Celery] main() [ Err Celery] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/celery/main.py", line 30, in main [ Err Celery] main() [ Err Celery] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/celery/bin/celery.py", line 81, in main [ Err Celery] cmd.execute_from_commandline(argv) [ Err Celery] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/celery/bin/celery.py", line 770, in execute_from_commandline [ Err Celery] super(CeleryCommand, self).execute_from_commandline(argv))) [ Err Celery] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/celery/bin/base.py", line 309, in execute_from_commandline [ Err Celery] argv = self.setup_app_from_commandline(argv) [ Err Celery] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/celery/bin/base.py", line 469, in setup_app_from_commandline [ Err Celery] self.app = self.find_app(app) [ Err Celery] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/celery/bin/base.py", line 489, in find_app [ Err Celery] return find_app(app, symbol_by_name=self.symbol_by_name) [ Err Celery] File "/home/usr/pq/tests/.hitch/py3.4.3/lib/python3.4/site-packages/celery/app/utils.py", line 240, in find_app [ Err Celery] found = sym.celery [ Err Celery] AttributeError: 'module' object has no attribute 'celery' [ Firefox] {"uri": "http://127.0.0.1:36785/hub"} [ Firefox] READY [ Hitch] Firefox Loaded. Python 3.4.3 (default, Sep 15 2015, 12:35:19) Type "copyright", "credits" or "license" for more information.

IPython 4.0.2 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details.

Exception occurred in "Stub" setup /home/usr/pq/tests/stub.test --tags feature-name

/home/usr/pq/tests/engine.py

100 :         )
101 :

--> 102 : self.services.startup(interactive=False) 103 : 104 : # Docs : https://hitchtest.readthedocs.org/en/latest/plugins/hitchcli.html

/home/usr/pq/tests/.hitch/virtualenv/lib/python3.4/site-packages/hitchserve/service_bundle.py

194 :             if interactive:
195 :                 self.start_interactive_mode()

--> 196 : raise what_happened 197 : else: 198 : if interactive:

hitchserve.hitch_exception.ServiceSuddenStopException

Service 'Celery' stopped suddenly.

In 1:

crdoconnor commented 8 years ago

This time the relevant error is here:

[ Err Celery] AttributeError: 'module' object has no attribute 'celery'

It looks like this might be relevant to the problem you're having: https://stackoverflow.com/questions/13555386/starting-celery-attributeerror-module-object-has-no-attribute-celery#13569129 (difficult to tell exactly without looking at the code).

crdoconnor commented 8 years ago

The command it's trying to run is:

/home/usr/pq/tests/.hitch/py3.4.3/bin/python -u -m celery worker --app=pq --loglevel=INFO --concurrency=2

In directory: /home/usr/pq

musicformellons commented 8 years ago

Ok, thanks. My celery setup is indeed non-existent. Hitch looks pretty impressive. Thanks for your swift help. Will get back on this later.

crdoconnor commented 8 years ago

No problem.

If you want to prevent hitch from running celery for now you can just comment out the lines that look like this:

 self.services['Celery'] = hitchpython.CeleryService(
        python=python_package.python,
        app="appname", loglevel="INFO",
        needs=[],
)

It'll then not attempt to start that service.