search

preset-io / elasticsearch-dbapi

A DBAPI and SQLAlchemy dialect for Elasticsearch
Apache License 2.0
108 stars 50 forks source link

OpenDistro - Index with nested columns throwing errors while exporting as Table Visualization #6

Open syamat opened 4 years ago

syamat commented 4 years ago

Index with nested columns throwing errors while exporting as Table Visualization. The data is shown correctly in SQL Lab.

:POST http://localhost:9200/_opendistro/_sql/?format=csv [status:N/A request:10.012s] Traceback (most recent call last): File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request six.raise_from(e, None) File "", line 2, in raise_from File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.6/http/client.py", line 1346, in getresponse response.begin() File "/usr/lib/python3.6/http/client.py", line 307, in begin version, status, reason = OpenDistro_Superset_Error_2 OpenDistro_Superset_Error_3

self._read_status() File "/usr/lib/python3.6/http/client.py", line 268, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib/python3.6/socket.py", line 586, in readinto return self._sock.recv_into(b) socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 238, in perform_request method, url, body, retries=Retry(False), headers=request_headers, *kw File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 641, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 344, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 603, in urlopen chunked=chunked) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 389, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 307, in _raise_timeout raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value) urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10) DEBUG:elasticsearch:> {"query":"SELECT name AS name, encounter AS encounter FROM (select from pt_idx) AS expr_qry LIMIT 1000"} ERROR:root:Query SELECT name AS name, encounter AS encounter FROM (select * from pt_idx) AS expr_qry LIMIT 1000 on schema default failed Traceback (most recent call last): File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request six.raise_from(e, None) File "", line 2, in raise_from File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.6/http/client.py", line 1346, in getresponse response.begin() File "/usr/lib/python3.6/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.6/http/client.py", line 268, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib/python3.6/socket.py", line 586, in readinto return self._sock.recv_into(b) socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 238, in perform_request method, url, body, retries=Retry(False), headers=request_headers, **kw File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 641, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 344, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise raise value File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 603, in urlopen chunked=chunked) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 389, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 307, in _raise_timeout raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value) urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/es/baseapi.py", line 220, in elastic_query resp = self.es.transport.perform_request("POST", path, body=payload) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/elasticsearch/transport.py", line 358, in perform_request timeout=timeout, File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 249, in perform_request raise ConnectionTimeout("TIMEOUT", str(e), e) elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/superset/connectors/sqla/models.py", line 999, in query df = self.database.get_df(sql, self.schema, mutator) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/superset/models/core.py", line 1009, in get_df self.db_engine_spec.execute(cursor, sqls[-1]) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/superset/db_engine_specs/base.py", line 773, in execute cursor.execute(query) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/es/baseapi.py", line 25, in g return f(self, *args, **kwargs) File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/es/opendistro/api.py", line 155, in execute _results = self.elastic_query(query, csv=True).split("\n") File "/usr/local/AT_Dev_Env/AmritaTech/apache-superset/master/incubator-superset/venv/lib/python3.6/site-packages/es/baseapi.py", line 223, in elastic_query f"Error connecting to {self.url}: {e.info}" es.exceptions.OperationalError: Error connecting to http://localhost:9200: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10) INFO:werkzeug:127.0.0.1 - - [20/Dec/2019 18:36:08] "POST /superset/explore_json/ HTTP/1.1" 400 - DEBUG:root:[stats_logger] (incr) log

syamat commented 4 years ago

OpenDistro_Superset_Error_3

dpgaspar commented 4 years ago

@syamat, sorry but opendistro is not fully supported yet, charts won't work

dai-chen commented 3 years ago

@dpgaspar FYI, ODFE 1.13 release is coming soon with new SQL query engine turned on. Here is the doc: https://github.com/opendistro-for-elasticsearch/sql/blob/develop/docs/dev/NewSQLEngine.md. Feel free to reach out to us if any question. We can also schedule meeting if needed. Thanks!

syamat commented 3 years ago

Hi @dai-chen

Will the new engine support nested fields? If not, will it fall-back to old engine in case a nested field is queried using the new SQL engine.

syamat commented 3 years ago

With this issue be resolved with the next release?

dai-chen commented 3 years ago

With this issue be resolved with the next release?

The new engine only supports object fields in ODFE 1.13. The nested field query will fall back to old engine.Please find more details here: https://github.com/opendistro-for-elasticsearch/sql/blob/develop/docs/user/beyond/partiql.rst#querying-nested-tuple-values. We will add support for nested fields in future release. Thanks!

syamat commented 3 years ago

Is this issue fixed with the new release?