CybercentreCanada / jupyterlab-sql-editor

A JupyterLab extension providing, SQL formatter, auto-completion, syntax highlighting, Spark SQL and Trino
BSD 3-Clause "New" or "Revised" License
83 stars 13 forks source link

Trino cell magic(%%trino) does not work while same query works with trino line magic(%trino) #54

Closed surjansr closed 2 years ago

surjansr commented 2 years ago

When using %%trino it always errors out with syntax error TrinoUserError: TrinoUserError(type=USER_ERROR, name=SYNTAX_ERROR, message="line 1:22: mismatched input '<EOF>'. Expecting: 'ALTER', 'ANALYZE', 'CALL', 'COMMENT', 'COMMIT', 'CREATE', 'DEALLOCATE', 'DELETE', 'DENY', 'DESC', 'DESCRIBE', 'DROP', 'EXECUTE', 'EXPLAIN', 'GRANT', 'INSERT', 'MERGE', 'PREPARE', 'REFRESH', 'RESET', 'REVOKE', 'ROLLBACK', 'SET', 'SHOW', 'START', 'TRUNCATE', 'UPDATE', 'USE', <query>", query_id=20220910_013206_00228_xf4wd)

image

using the same query with line magic works well which means the connections and other params are fine.

Inspecting the query being sent by %%trino , I see that the sql query is not even being passed to trino.

image

Logs

---------------------------------------------------------------------------
TrinoUserError                            Traceback (most recent call last)
/tmp/ipykernel_48/3498095607.py in <module>
----> 1 get_ipython().run_cell_magic('trino', '--output html', "select 'hello line magix' as test\n")

~/.local/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
   2470             with self.builtin_trap:
   2471                 args = (magic_arg_s, cell)
-> 2472                 result = fn(*args, **kwargs)
   2473             return result
   2474 

~/.local/lib/python3.7/site-packages/decorator.py in fun(*args, **kw)
    230             if not kwsyntax:
    231                 args, kw = fix(args, kw, sig)
--> 232             return caller(func, *(extras + args), **kw)
    233     fun.__name__ = func.__name__
    234     fun.__doc__ = func.__doc__

~/.local/lib/python3.7/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188 
    189         if callable(arg):

~/.local/lib/python3.7/site-packages/jupyterlab_sql_editor/ipython_magic/trino/trino.py in trino(self, cell, line, local_ns)
    114 
    115         self.cur.execute(sql)
--> 116         results = self.cur.fetchmany(limit+1)
    117 
    118         columns = list(map(lambda d: d[0], self.cur.description))

~/.local/lib/python3.7/site-packages/trino/dbapi.py in fetchmany(self, size)
    556         result = []
    557         for _ in range(size):
--> 558             row = self.fetchone()
    559             if row is None:
    560                 break

~/.local/lib/python3.7/site-packages/trino/dbapi.py in fetchone(self)
    524         try:
    525             assert self._iterator is not None
--> 526             return next(self._iterator)
    527         except StopIteration:
    528             return None

~/.local/lib/python3.7/site-packages/trino/client.py in __iter__(self)
    612         # Subsequent fetches from GET requests until next_uri is empty.
    613         while not self._query.finished:
--> 614             rows = self._query.fetch()
    615             for row in rows:
    616                 self._rownumber += 1

~/.local/lib/python3.7/site-packages/trino/client.py in fetch(self)
    789         """Continue fetching data for the current query_id"""
    790         response = self._request.get(self._request.next_uri)
--> 791         status = self._request.process(response)
    792         self._update_state(status)
    793         logger.debug(status)

~/.local/lib/python3.7/site-packages/trino/client.py in process(self, http_response)
    534         logger.debug("HTTP %s: %s", http_response.status_code, response)
    535         if "error" in response:
--> 536             raise self._process_error(response["error"], response.get("id"))
    537 
    538         if constants.HEADER_CLEAR_SESSION in http_response.headers:

TrinoUserError: TrinoUserError(type=USER_ERROR, name=SYNTAX_ERROR, message="line 1:23: mismatched input '<EOF>'. Expecting: 'ALTER', 'ANALYZE', 'CALL', 'COMMENT', 'COMMIT', 'CREATE', 'DEALLOCATE', 'DELETE', 'DENY', 'DESC', 'DESCRIBE', 'DROP', 'EXECUTE', 'EXPLAIN', 'GRANT', 'INSERT', 'MERGE', 'PREPARE', 'REFRESH', 'RESET', 'REVOKE', 'ROLLBACK', 'SET', 'SHOW', 'START', 'TRUNCATE', 'UPDATE', 'USE', <query>", query_id=20220910_013656_00235_xf4wd)
surjansr commented 2 years ago

@cccs-jc I think I found the reason for the above bug. the spark SQL and trino magic both are defined to be line_cell_magic but the order of the arguments is different and hence trino is taking only the line argument when executing and ignoring the cell content and we are getting "No sql statement to execute"

def trino(self, cell=None, line=None, local_ns=None): 
        "Magic that works both as %trino and as %%trino"
        self.set_user_ns(local_ns)
        args = parse_argstring(self.trino, line) 
def sparksql(self, line=None, cell=None, local_ns=None):
        "Magic that works both as %sparksql and as %%sparksql"
        self.set_user_ns(local_ns)
        args = parse_argstring(self.sparksql, line)
        output_file = self.outputFile or f"{os.path.expanduser('~')}/.local/sparkdb.schema.json"
        output = args.output.lower()
image

@cccs-jc @cccs-nik : possible to put the fix in for this. For me the approval process will take time.

cccs-nik commented 2 years ago

Sorry about that, I accidentally messed up the order of the args. Thanks for bringing up the issue and the fix 🙂Fix is merged.