installation on ubuntu 10.10 server problem: celeryd config file missing

[ecostat]

Hi there

I followed the manual installation instructions on a fresh ubuntu 10.10 server and made it to the last step under (Running OpenQuake)

... cd /to/your/openquake/dir/ celeryd

However celeryd won't start because it cannot find the celeryconfig.py file - which nevertheless - resides within the same directory. I also cannot find the default setup script /etc/default/celeryd

Any ideas ?

See call and installation-log below

---

heiri@sepp:~/openquake$ celeryd /usr/local/lib/python2.6/dist-packages/celery/loaders/default.py:53: NotConfigured: No celeryconfig.py module found! Please make sure it exists and is available to Python. NotConfigured) Traceback (most recent call last): File "/usr/local/bin/celeryd", line 9, in load_entry_point('celery==2.1.4', 'console_scripts', 'celeryd')() File "/usr/local/lib/python2.6/dist-packages/celery/bin/celeryd.py", line 166, in main worker.execute_from_commandline() File "/usr/local/lib/python2.6/dist-packages/celery/bin/base.py", line 40, in execute_from_commandline return self.run(_args, *_vars(options)) File "/usr/local/lib/python2.6/dist-packages/celery/bin/celeryd.py", line 85, in run return Worker(**kwargs).run() File "/usr/local/lib/python2.6/dist-packages/celery/apps/worker.py", line 98, in run self.init_loader() File "/usr/local/lib/python2.6/dist-packages/celery/apps/worker.py", line 147, in init_loader "Celery needs to be configured to run celeryd.") celery.exceptions.ImproperlyConfigured: Celery needs to be configured to run celeryd.

---

heiri@sepp:~$ sudo easy_install celery install_dir /usr/local/lib/python2.6/dist-packages/ Searching for celery Best match: celery 2.1.4 celery 2.1.4 is already the active version in easy-install.pth Installing celeryctl script to /usr/local/bin Installing celeryd script to /usr/local/bin Installing camqadm script to /usr/local/bin Installing celeryev script to /usr/local/bin Installing celeryd-multi script to /usr/local/bin Installing celerybeat script to /usr/local/bin

[larsbutler]

Hi ecostat,

Can please you provide us with the following info?

• Output of $ echo $PATH
• Output of $ echo $PYTHONPATH
• Directory contents of ~/openquake

Also, from ~/openquake, can you try to 'import celeryconfig' in a python console? Please report any warnings or errors that occur when you attempt this.

I don't know immediately what the problem is, but this should help us narrow down the possible causes.

Also, feel free to join our IRC channel (#openquake on irc.freenode.net) if you'd like to get direct help from the team. Please keep in mind, however, that most of our team members are in Italy and Switzerland (GMT +1).

Thanks, Lars

[ecostat]

Hi Lars,

thanks for troubleshooting. Right, I haven't set explicitly any environment variables so far. PYTHONPATH is empty. As a newbie, I first wanted to get openquake running and play with it before I dwelve into the python stuff (I am from the java corner). Seems like playing around doesn't come for free. See below:

regards ecostat

1) heiri@sepp:~$ echo $PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

2) heiri@sepp:~$ echo $PYTHONPATH

heiri@sepp:~$

3) heiri@sepp:~/openquake$ python Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39) [GCC 4.4.5] on linux2 Type "help", "copyright", "credits" or "license" for more information. ">>>" import celeryconfig

Traceback (most recent call last): File "", line 1, in File "celeryconfig.py", line 11, in from openquake import flags File "openquake/flags.py", line 9, in from gflags import * File "/usr/local/lib/python2.6/dist-packages/gflags.py", line 396, in import gflags_validators ImportError: No module named gflags_validators

4) heiri@sepp:~/openquake$ ls bin install.sh README.txt build java requirements.txt build.xml java_tests run_tests.py celeryconfig.py lib setup.py celeryconfig.pyc LICENSE smoketest.py demo_converter.sh log4j.properties smoketests demo_risk.sh loss-curves-jobber.xml sql demo_server.sh MANIFEST.in templates demo_smoketest.sh mixin_demo.py tests dist openquake tools docs pylintrc fabfile.py quake.tmproj

[larsbutler]

Hey ecostat,

Thanks for the info. It looks like the source of the issue is python-gflags.

Can you do a $ pip freeze and pastebin the results? I'd like to see the versions of the python packages you have installed.

Here is a snapshot of my configuration for comparison: http://pastebin.com/kdvR58f7. (Please note there are some irrelevant packages listed here.)

If your version of python-gflags is < 1.4, you can try upgrading it: $ sudo pip install python-gflags==1.4

If you already have >= 1.4 installed, try $ sudo pip uninstall python-gflags, then install it gain.

Let me know how that goes.

Cheers, Lars

[ecostat]

Hi Lars,

Good nose ! I had already python-gflags==1.5 and after reinstalling the same version I could start celeryd.

running smoke test:

The 'smoke test' runs through and delivers several tiff, html, svg files. The html legends look plausible, but when I look at the corresponding tiff they are all very very green - no color gradient visible. Already a big step for me, but I guess not for mankind ;-)

running test suite:

was less successfull. I get a couple of error messages and kind of abort. The output is extremely verbose I pasted just some lines at the end of the log below.

possible causes ?

a) Looking at the celeryd output at startup I wonder whether 'localhost' should be part of the amqp connect string (see my /etc/hosts file below) , or a WARNING: heartbeat is disabled is a good sign ?

b) with an Ubuntu 10.10 32bit installation I would not have expected something like 'TypeError: 'numpy.float64' object is not iterable'

anyway thanks a lot so far I guess I now have to read a bit of documentation. Eventually you just confirm that these issues don't have anything to do with my celeryd-setup so that I can close this issue.

best regards ecostat

---

LOG

---

heiri@sepp:~$ pip freeze Django==1.2.3 Fabric==0.9.0 GDAL==1.6.3 GnuPGInterface==0.3.2 JPype==0.5.4.1 Jinja2==2.5.5 PyYAML==3.09 Pygments==1.4 Shapely==1.2.1 Sphinx==1.0.7 amqplib==0.6.1 anyjson==0.3 apt-xapian-index==0.39 carrot==0.10.7 celery==2.1.4 command-not-found==0.1 distribute==0.6.14 docutils==0.7 guppy==0.1.9 importlib==1.0.2 lxml==2.2.6 matplotlib==0.99.3 nose==0.11.1 numpy==1.3.0 paramiko==1.7.6 pycrypto==2.0.1 pyparsing==1.5.2 python-apt==0.7.96.1ubuntu11 python-dateutil==1.4.1 python-debian==0.1.16ubuntu1 python-geohash==0.7.1 python-gflags==1.5 pytz==2010b redis==2.0.0 scipy==0.7.2 ufw==0.30.0-1ubuntu2 unattended-upgrades==0.1 wsgiref==0.1.2

---

heiri@sepp:~$ sudo pip install python-gflags --upgrade Downloading/unpacking python-gflags Downloading python-gflags-1.5.1.tar.gz (58Kb): 58Kb downloaded Running setup.py egg_info for package python-gflags Installing collected packages: python-gflags Found existing installation: python-gflags 1.5 Uninstalling python-gflags: Successfully uninstalled python-gflags Running setup.py install for python-gflags Successfully installed python-gflags Cleaning up...

---

heiri@sepp:~/openquake$ celeryd /usr/lib/pymodules/python2.6/jpype/_pykeywords.py:18: DeprecationWarning: the sets module is deprecated import sets [2011-02-02 20:29:53,249: WARNING/MainProcess] celery@sepp v2.1.4 is starting. [2011-02-02 20:29:53,266: WARNING/MainProcess]
Configuration -> . broker -> amqp://celeryuser@localhost:5672/celeryvhost . queues -> . celery -> exchange:celery (direct) binding:celery . concurrency -> 1 . loader -> celery.loaders.default.Loader . logfile -> [stderr]@WARNING . events -> OFF . beat -> OFF

[2011-02-02 20:29:53,428: WARNING/MainProcess] celery@sepp has started. [2011-02-02 20:33:50,155: WARNING/PoolWorker-1] Opening a new redis connection [2011-02-02 20:34:01,403: WARNING/PoolWorker-1] 2011-02-02 20:34:01,400 INFO [JREDIS] - WARNING: heartbeat is disabled. ... ...

---

heiri@sepp:~$ more /etc/hosts 127.0.0.1 localhost 127.0.1.1 sepp sepp.ecostat.ch The following lines are desirable for IPv6 capable hosts ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters

---

heiri@sepp:~/openquake$ python run_tests.py /usr/lib/pymodules/python2.6/jpype/_pykeywords.py:18: DeprecationWarning: the sets module is deprecated import sets .DEBUG:root:Found 3 stored loss curves... DEBUG:root:Aggregate loss curve stored at /home/heiri/openquake/tests/data/output/1234-aggregate-loss-curve.svg .DEBUG:root:AGGREGATE_LOSS_CURVE parameter not specified, skipping aggregate loss curve computation......SDEBUG:root:Found data at /home/heiri/openquake/tests/data/ExposurePortfolioFile-test.xml ... '============================================================= 'FAIL: test_end_to_end (tests.QuantileHazardCurveComputationTestCase) '---------------------------------------------------------------------- Traceback (most recent call last): File "/home/heiri/openquake/tests/hazard_unittest.py", line 662, in test_end_to_end kvs.tokens.QUANTILE_HAZARD_CURVE_KEY_TOKEN, engine.id))) > 0) AssertionError: -------------------- >> begin captured stdout << --------------------- source model key is d30c32c4!sources!! GMPE map key is d30c32c4!gmpe!!

--------------------- >> end captured stdout << ---------------------- -------------------- >> begin captured logging << -------------------- root: DEBUG: Loading Job from /home/heiri/openquake/smoketests/classical_psha_simple/config.gem root: WARNING: No default configuration! If your job config doesn't define all of the expected properties things might break. root: DEBUG: Slurping /home/heiri/openquake/smoketests/classical_psha_simple/source_model_logic_tree.xml root: DEBUG: Slurping /home/heiri/openquake/smoketests/classical_psha_simple/gmpe_logic_tree.xml root: INFO: Going to run classical PSHA hazard for 1 realizations and 91 sites root: INFO: Calculatiing hazard curves for realization 0 root: INFO: Storing source model from job config --------------------- >> end captured logging << ---------------------

---

Ran 206 tests in 142.551s

FAILED (SKIP=6, failures=1)

---

heiri@sepp:~/openquake$ celeryd /usr/lib/pymodules/python2.6/jpype/_pykeywords.py:18: DeprecationWarning: the sets module is deprecated import sets [2011-02-02 20:29:53,249: WARNING/MainProcess] celery@sepp v2.1.4 is starting. [2011-02-02 20:29:53,266: WARNING/MainProcess]
Configuration -> broker -> amqp://celeryuser@localhost:5672/celeryvhost queues -> celery -> exchange:celery (direct) binding:celery concurrency -> 1 loader -> celery.loaders.default.Loader logfile -> [stderr]@WARNING events -> OFF beat -> OFF

[2011-02-02 20:29:53,428: WARNING/MainProcess] celery@sepp has started. [2011-02-02 20:33:50,155: WARNING/PoolWorker-1] Opening a new redis connection [2011-02-02 20:34:01,403: WARNING/PoolWorker-1] 2011-02-02 20:34:01,400 INFO [JREDIS] - WARNING: heartbeat is disabled. ... ... [2011-02-02 20:35:17,608: ERROR/MainProcess] Task openquake.hazard.tasks.compute_mean_curves[bb90514a-cf6c-4ef3-acf8-c3d522c60797] raised exception: TypeError("'numpy.float64' object is not iterable",) Traceback (most recent call last): File "/usr/local/lib/python2.6/dist-packages/celery/execute/trace.py", line 34, in trace return cls(states.SUCCESS, retval=fun(_args, _kwargs)) File "/usr/local/lib/python2.6/dist-packages/celery/task/base.py", line 248, in call return self.run(_args, _kwargs) File "/usr/local/lib/python2.6/dist-packages/celery/decorators.py", line 52, in run return fun(_args, *_kwargs) File "/home/heiri/openquake/openquake/hazard/tasks.py", line 104, in compute_mean_curves classical_psha.compute_mean_hazard_curves(job_id, sites) File "/home/heiri/openquake/openquake/hazard/classical_psha.py", line 98, in compute_mean_hazard_curves "curve": list(compute_mean_curve(curves_at(job_id, site)))} TypeError: 'numpy.float64' object is not iterable

None [2011-02-02 20:35:38,199: WARNING/PoolWorker-1] 2011-02-02 20:35:38,199 INFO [JREDIS] - WARNING: heartbeat is disabled.

[larsbutler]

Cool, looks like we're getting there.

Regarding your questions:

a) The 'WARNING: heartbeat is disabled.' message is normal. It is safe to ignore. =)

b) One of my colleagues (Andrea) believes that the error you are experiencing is due to a fragile test. He suggests pulling the latest code from the github repo ($ git pull origin master) to fix the problem.

Let me know if that helps.

-Lars

[ecostat]

Hi Lars,

I did a 'git pull origin master' and it helped partially: The 'TypeError: 'numpy.float64' disappeared - no red error messages any more.

At some stage it crashes, telling me again that it ran 206 test. See tail of the output below, it is slightly different from the one before. What about this 'INFO:paramiko.transport:Authentication (password) failed. ' ?

-ecostat

'*****

LOG

DEBUG:paramiko.transport:Attempting password auth... DEBUG:paramiko.transport:userauth is OK INFO:paramiko.transport:Authentication (password) failed. DEBUG:paramiko.transport:EOF in transport thread ..............DEBUG:root:Found 3 stored loss curves... DEBUG:root:Aggregate loss curve stored at /home/heiri/openquake/tests/data/output/1234-aggregate-loss-curve.svg DEBUG:root:AGGREGATE_LOSS_CURVE parameter not specified, skipping aggregate loss curve computation............................ DEBUG:root:Found 3 stored loss curves...

---

Ran 206 tests in 197.901s

OK (SKIP=6) Exception ImportError: 'No module named shapely.geos' in <bound method Point.del of <shapely.geometry.point.Point object at 0xa273d2c>> ignored Exception ImportError: 'No module named shapely.geos' in <bound method Point.del of <shapely.geometry.point.Point object at 0xa273ccc>> ignored Exception ImportError: 'No module named shapely.geos' in <bound method Point.del of <shapely.geometry.point.Point object at 0xa273d6c>> ignored Exception ImportError: 'No module named shapely.geos' in <bound method Point.del of <shapely.geometry.point.Point object at 0xa273bec>> ignored

[larsbutler]

Hey ecostat,

The message about authentication is simply debug output from a test which exercises a fail condition. So everything is fine there.

I'm not sure about the import error at the bottom, but it doesn't appear to be fatal. I see that you have Shapely==1.2.1... a few of the developers here are running with the same version and they see the same message. So I think you can ignore it for now, though the solution may be to upgrade to a newer version (I have Shapely==1.2.6 and do not see this message).

Technically, all of the tests ran fine (indicated by the 'OK'), so I think you're all set. Let us know if you have any more questions.

Cheers, Lars

[ecostat]

Hello Lars, no more questions for the moment, thanks for explanations. case closed. -ecostat