maggienj / ActiveData

Provide high speed filtering and aggregation over data
Mozilla Public License 2.0
0 stars 0 forks source link

Set-up question on Python Flask #56

Closed javatreble closed 7 years ago

javatreble commented 7 years ago

@maggienj

Do I need to download Flask separately? I get this error message. ImportError: No module named flask

====================================================================== ERROR: test_sql (unittest.loader.ModuleImportFailure)

ImportError: Failed to import test module: test_sql Traceback (most recent call last): File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/loader.py", line 254, in _find_tests module = self._get_module_from_name(name) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name import(name) File "/Users/lmilton/PycharmProjects/ActiveData/tests/test_sql.py", line 20, in from tests import compare_to_expected File "tests/init.py", line 21, in from active_data.actions.jx import replace_vars File "active_data/init.py", line 13, in import flask ImportError: No module named flask

====================================================================== ERROR: test_unittests (unittest.loader.ModuleImportFailure)

ImportError: Failed to import test module: test_unittests Traceback (most recent call last): File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/loader.py", line 254, in _find_tests module = self._get_module_from_name(name) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name import(name) File "/Users/lmilton/PycharmProjects/ActiveData/tests/test_unittests.py", line 23, in from pyLibrary.env import http File "pyLibrary/env/http.py", line 28, in from requests import sessions, Response ImportError: No module named requests

maggienj commented 7 years ago

no , you don't have to download flask separately. it is all part of the requirements.txt it should download by itself. you have to run one command .... which will download all modules that are required .

maggienj commented 7 years ago

pip install -r requirements.txt

(copied from https://github.com/klahnakoski/ActiveData/tree/master )

lmilton-ithaka commented 7 years ago

Update: Never mind. Running with sudo command fixed it.

Thanks @maggienj Maggie . I got the following message related to permissions on my computer.

Requirement already satisfied: pyparsing in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from moz-sql-parser->-r requirements.txt (line 4)) Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->Flask->-r requirements.txt (line 1)) Downloading MarkupSafe-1.0.tar.gz Installing collected packages: itsdangerous, Werkzeug, MarkupSafe, Jinja2, click, Flask, pymysql, urllib3, idna, certifi, chardet, requests, moz-sql-parser Running setup.py install for itsdangerous ... error Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/sb/vy309mss1g11hz3y6h1jqzwx5ggrc1/T/pip-build-WWTL_K/itsdangerous/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /var/folders/sb/vy309mss1g11hz3y6h1jqzwx5ggrc1/T/pip-coTqjv-record/install-record.txt --single-version-externally-managed --compile: running install running build running build_py creating build creating build/lib copying itsdangerous.py -> build/lib running install_lib copying build/lib/itsdangerous.py -> /Library/Python/2.7/site-packages error: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/itsdangerous.py'

----------------------------------------

Command "/usr/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/sb/vy309mss1g11hz3y6h1jqzwx5ggrc1/T/pip-build-WWTL_K/itsdangerous/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /var/folders/sb/vy309mss1g11hz3y6h1jqzwx5ggrc1/T/pip-coTqjv-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/sb/vy309mss1g11hz3y6h1jqzwx5ggrc1/T/pip-build-WWTL_K/itsdangerous/

lmilton-ithaka commented 7 years ago

I am at this step now relating to setup in command line.

caused by ERROR: Tried 1 times: Request failure of http://127.0.0.1:5000/sql File "pyLibrary/env/http.py", line 155, in request File "pyLibrary/env/http.py", line 200, in post File "tests/init.py", line 259, in post_till_response File "/Users/lmilton/PycharmProjects/ActiveData/tests/test_sql.py", line 92, in _run_sql_query File "/Users/lmilton/PycharmProjects/ActiveData/tests/test_sql.py", line 73, in test_select_from_dual File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 331, in run File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 395, in call File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 108, in run File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 70, in call File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 108, in run File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 70, in call File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 108, in run File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 70, in call File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/runner.py", line 151, in run File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 232, in runTests File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 95, in init File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 12, in File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main caused by ERROR: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /sql (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x108fbf690>: Failed to establish a new connection: [Errno 61] Connection refused',)) File "/Library/Python/2.7/site-packages/requests/adapters.py", line 504, in send File "/Library/Python/2.7/site-packages/requests/sessions.py", line 612, in send File "/Library/Python/2.7/site-packages/requests/sessions.py", line 502, in request File "pyLibrary/env/http.py", line 148, in request File "pyLibrary/env/http.py", line 200, in post File "tests/init.py", line 259, in post_till_response File "/Users/lmilton/PycharmProjects/ActiveData/tests/test_sql.py", line 92, in _run_sql_query File "/Users/lmilton/PycharmProjects/ActiveData/tests/test_sql.py", line 73, in test_select_from_dual File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 331, in run File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 395, in call File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 108, in run File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 70, in call File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 108, in run File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 70, in call File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 108, in run File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 70, in call File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/runner.py", line 151, in run File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 232, in runTests File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 95, in init File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 12, in File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main


Ran 224 tests in 26.316s

FAILED (errors=172, skipped=16)

lmilton-ithaka commented 7 years ago

@maggienj While running this command, I get this error.

export PYTHONPATH=. python active_data/app.py --settings=resources/config/simple_settings.json

ITHPR1LMILTON (pid 9912) - 2017-07-21 21:45:49 - Main Thread - "app.py:112" (setup) - WARNING: ActiveData is in debug mode File "active_data/app.py", line 112, in setup File "active_data/app.py", line 195, in

ITHPR1LMILTON (pid 9912) - 2017-07-21 21:45:49 - Main Thread - "elasticsearch.py:645" (create_index) - WARNING: Reduced number of replicas: 2 requested, 0 realized File "/Users/lmilton/PycharmProjects/ActiveData/pyLibrary/env/elasticsearch.py", line 645, in create_index File "/Users/lmilton/PycharmProjects/ActiveData/mo_kwargs/init.py", line 71, in wrapper File "/Users/lmilton/PycharmProjects/ActiveData/pyLibrary/env/elasticsearch.py", line 526, in get_or_create_index File "/Users/lmilton/PycharmProjects/ActiveData/mo_kwargs/init.py", line 71, in wrapper File "/Users/lmilton/PycharmProjects/ActiveData/active_data/actions/save_query.py", line 73, in init File "/Users/lmilton/PycharmProjects/ActiveData/mo_kwargs/init.py", line 64, in wrapper File "active_data/app.py", line 118, in setup File "active_data/app.py", line 195, in

Traceback (most recent call last): File "active_data/app.py", line 195, in setup() File "active_data/app.py", line 128, in setup Log.error("Serious problem with ActiveData service construction! Shutdown!", cause=e) File "/Users/lmilton/PycharmProjects/ActiveData/mo_logs/init.py", line 392, in error raise e mo_logs.exceptions.Except: ERROR: Serious problem with ActiveData service construction! Shutdown! File "active_data/app.py", line 128, in setup File "active_data/app.py", line 195, in caused by ERROR: Problem with call to http://localhost:9200/saved_queries20170721_214549 File "/Users/lmilton/PycharmProjects/ActiveData/pyLibrary/env/elasticsearch.py", line 867, in put File "/Users/lmilton/PycharmProjects/ActiveData/pyLibrary/env/elasticsearch.py", line 652, in create_index File "/Users/lmilton/PycharmProjects/ActiveData/mo_kwargs/init.py", line 71, in wrapper File "/Users/lmilton/PycharmProjects/ActiveData/pyLibrary/env/elasticsearch.py", line 526, in get_or_create_index File "/Users/lmilton/PycharmProjects/ActiveData/mo_kwargs/init.py", line 71, in wrapper File "/Users/lmilton/PycharmProjects/ActiveData/active_data/actions/save_query.py", line 73, in init File "/Users/lmilton/PycharmProjects/ActiveData/mo_kwargs/init.py", line 64, in wrapper File "active_data/app.py", line 118, in setup File "active_data/app.py", line 195, in caused by ERROR: Bad Request: {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"Mapping definition for [query] has unsupported parameters: [store : true]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [default]: Mapping definition for [query] has unsupported parameters: [store : true]","caused_by":{"type":"mapper_parsing_exception","reason":"Mapping definition for [query] has unsupported parameters: [store : true]"}},"status":400} File "/Users/lmilton/PycharmProjects/ActiveData/pyLibrary/env/elasticsearch.py", line 849, in put File "/Users/lmilton/PycharmProjects/ActiveData/pyLibrary/env/elasticsearch.py", line 652, in create_index File "/Users/lmilton/PycharmProjects/ActiveData/mo_kwargs/init.py", line 71, in wrapper File "/Users/lmilton/PycharmProjects/ActiveData/pyLibrary/env/elasticsearch.py", line 526, in get_or_create_index File "/Users/lmilton/PycharmProjects/ActiveData/mo_kwargs/init.py", line 71, in wrapper File "/Users/lmilton/PycharmProjects/ActiveData/active_data/actions/save_query.py", line 73, in init File "/Users/lmilton/PycharmProjects/ActiveData/mo_kwargs/init.py", line 64, in wrapper File "active_data/app.py", line 118, in setup File "active_data/app.py", line 195, in

lmilton-ithaka commented 7 years ago

From inside PyCharm IDE, I get this /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 /Users/lmilton/PycharmProjects/ActiveData/active_data/app.py Traceback (most recent call last): File "/Users/lmilton/PycharmProjects/ActiveData/active_data/app.py", line 33, in from active_data.actions.json import get_raw_json File "/Users/lmilton/PycharmProjects/ActiveData/active_data/actions/json.py", line 19, in from active_data.actions.jx import send_error File "/Users/lmilton/PycharmProjects/ActiveData/active_data/actions/jx.py", line 28, in BLANK = convert.unicode2utf8(File("active_data/public/error.html").read()) File "/Users/lmilton/PycharmProjects/ActiveData/mo_files/init.py", line 186, in read with open(self._filename, b"rb") as f: IOError: [Errno 2] No such file or directory: u'active_data/public/error.html'

Process finished with exit code 1

maggienj commented 7 years ago

what is your system? windows or mac or unix?

lmilton-ithaka commented 7 years ago

mac

klahnakoski commented 7 years ago

@lmilton-ithaka When running tests for ActiveData, you will have three things running: Elasticsearch, ActiveData (the server) and the ActiveData tests.

Is ES working? curl http://localhost:9200 (or visit that location with your browser).

maggienj commented 7 years ago

i have the same err in my mac too. ( i tried the same in my mac a month ago and got the same err )

klahnakoski commented 7 years ago

@lmilton-ithaka also, what version of ActiveData are you attempting to run? If you want to have a working version of ActiveData, then use the master branch with ES version 1.7.x. If you are following @maggienj 's work, then use the es5 branch with ES version 5.x. The errors you pasted seem to indicate that there is a version mismatch

lmilton-ithaka commented 7 years ago

@klahnakoski & @maggienj . Thank you!

How do I run Active Data Server? I brought elastic search up now and I am on es5 branch.

lmilton@ITHPR1LMILTON ◆ 2017-07-21 ◆ 06:49 PM ◆ ~ cd PycharmProjects/ActiveData/ lmilton@ITHPR1LMILTON ◆ 2017-07-21 ◆ 06:50 PM ◆ ~/PycharmProjects/ActiveData ls LICENSE contribute.json jx_base mo_collections mo_json mo_logs mo_threads pyLibrary tests README.md docs jx_elasticsearch mo_dots mo_json_config mo_math mo_times requirements.txt active_data examples jx_python mo_files mo_kwargs mo_testing moz_sql_parser resources lmilton@ITHPR1LMILTON ◆ 2017-07-21 ◆ 06:50 PM ◆ ~/PycharmProjects/ActiveData git status On branch es5 Your branch is up-to-date with 'origin/es5'. nothing to commit, working directory clean lmilton@ITHPR1LMILTON ◆ 2017-07-21 ◆ 06:50 PM ◆ ~/PycharmProjects/ActiveData curl http://localhost:9200/ { "name" : "A94b5EQ", "cluster_name" : "lavanya-application", "cluster_uuid" : "uj3uzhSlSQyEHoB4oKM_SA", "version" : { "number" : "5.2.2", "build_hash" : "f9d9b74", "build_date" : "2017-02-24T17:26:45.835Z", "build_snapshot" : false, "lucene_version" : "6.4.1" }, "tagline" : "You Know, for Search" } lmilton@ITHPR1LMILTON ◆ 2017-07-21 ◆ 06:50 PM ◆ ~/PycharmProjects/ActiveData

klahnakoski commented 7 years ago

Looks good so far, which is bad for you because it does not actually work.

The ActiveData server is run with this command:

python active_data/app.py --settings=resources/config/simple_settings.json

I am still looking at your logs....

klahnakoski commented 7 years ago

I have a theory: The saved_queries index is not being created because the schema is wrong. Maggie and I already have the index setup, so we have not noticed this bug....

klahnakoski commented 7 years ago

Yes, that is it. Fixing....

BTW, the real server start instruction must point to an ES5 friendly config file:

python active_data/app.py --settings=tests/es5/kyle_app_config.json
klahnakoski commented 7 years ago

Done. Pull the changes and run the server

klahnakoski commented 7 years ago

...on es5 branch

javatreble commented 7 years ago

Ok, thanks. Let me try that.

javatreble commented 7 years ago

That worked. Thanks.

javatreble commented 7 years ago

python active_data/app.py --settings=resources/config/simple_settings.json

ITHPR1LMILTON (pid 11007) - 2017-07-21 23:40:49 - Main Thread - "elasticsearch.py:645" (create_index) - WARNING: Reduced number of replicas: 2 requested, 0 realized File "/Users/lmilton/PycharmProjects/ActiveData/pyLibrary/env/elasticsearch.py", line 645, in create_index File "/Users/lmilton/PycharmProjects/ActiveData/mo_kwargs/init.py", line 73, in wrapper File "/Users/lmilton/PycharmProjects/ActiveData/pyLibrary/env/elasticsearch.py", line 526, in get_or_create_index File "/Users/lmilton/PycharmProjects/ActiveData/mo_kwargs/init.py", line 73, in wrapper File "/Users/lmilton/PycharmProjects/ActiveData/active_data/actions/save_query.py", line 73, in init File "/Users/lmilton/PycharmProjects/ActiveData/mo_kwargs/init.py", line 66, in wrapper File "active_data/app.py", line 118, in setup File "active_data/app.py", line 195, in

ITHPR1LMILTON (pid 11007) - 2017-07-21 23:40:49 - Main Thread - "elasticsearch.py:665" (create_index) -


** Made new index "saved_queries20170721_234049"


ITHPR1LMILTON (pid 11007) - 2017-07-21 23:40:49 - Main Thread - "elasticsearch.py:120" (init) -


** elasticsearch debugging for http://localhost:9200/saved_queries20170721_234049/query is on


ITHPR1LMILTON (pid 11007) - 2017-07-21 23:40:49 - Main Thread - "elasticsearch.py:300" (extend) - Timer start: Add 1.0 documents to saved_queries20170721_234049 ITHPR1LMILTON (pid 11007) - 2017-07-21 23:40:49 - Main Thread - "elasticsearch.py:352" (extend) - Timer end : Add 1.0 documents to saved_queries20170721_234049 (took 0.075 seconds) ITHPR1LMILTON (pid 11007) - 2017-07-21 23:40:49 - Main Thread - "elasticsearch.py:120" (init) -


** elasticsearch debugging for http://localhost:9200/saved_queries20170721_234049/query is on


maggienj commented 7 years ago

ActiveData is up and running for @javatreble and @lmilton-ithaka So, closing this.