search

openedx-vlead / port-labs-to-openedx

3 stars 8 forks source link

Remote task error #19

Closed sivashankerN closed 8 years ago

sivashankerN commented 8 years ago

We were facing the problem while running the remote tasks example task as follows

remote-task --host localhost --repo https://github.com/edx/edx-analytics-pipeline --user ubuntu --override-config $HOME/edx-analytics-pipeline/config/devstack.cfg --wheel-url http://edx-wheelhouse.s3-website-us-east-1.amazonaws.com/Ubuntu/precise --remote-name analyticstack --wait TotalEventsDailyTask --interval 2015 --output-root hdfs://localhost:9000/output/ --local-scheduler

Output of above task was

Running command = ['ssh', '-tt', '-o', 'ForwardAgent=yes', '-o', 'StrictHostKeyChecking=no', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'KbdInteractiveAuthentication=no', '-o', 'PasswordAuthentication=no', '-o', 'User=root', '-o', 'ConnectTimeout=10', 'localhost', "sudo -Hu hadoop /bin/bash -c 'cd /var/lib/analytics-tasks/analyticstack/repo && . $HOME/.bashrc && /var/lib/analytics-tasks/analyticstack/venv/bin/launch-task TotalEventsDailyTask --interval 2015 --output-root hdfs://localhost:9000/output/ --local-scheduler --overwrite'"] Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts. Traceback (most recent call last): File "/var/lib/analytics-tasks/analyticstack/venv/bin/launch-task", line 6, in from edx.analytics.tasks.launchers.local import main File "/var/lib/analytics-tasks/analyticstack/venv/local/lib/python2.7/site-packages/edx/analytics/tasks/launchers/local.py", line 16, in import cjson ImportError: /var/lib/analytics-tasks/analyticstack/venv/local/lib/python2.7/site-packages/cjson.so: undefined symbol: PyUnicodeUCS2_DecodeUnicodeEscape Connection to localhost closed. Exiting with status = 1

sivashankerN commented 8 years ago

Solved by doing following steps

-download cjson from https://pypi.python.org/pypi/python-cjson/1.0.5 -extract the files and execute ./build_inplace to compile the library -replace cjson.so file in /var/lib/analytics-tasks/analyticstack/venv/local/lib/python2.7/site-packages

-in the analytic stack venv, reinstall numpy and pandas:

cd /var/lib/analytics-tasks/analyticstack

. venv/bin/activate

pip uninstall numpy

pip install numpy==1.8.0

pip uninstall pandas

pip install pandas==0.13

Reference https://groups.google.com/forum/#!topic/openedx-analytics/51f8zWpcvAA