Closed ebatsell closed 5 years ago
I have tried this on Impala and does not seem to work.
I locally imported your version of pyverdict:
>>> from pyverdict import VerdictContext as vc
>>> conn = vc.new_impala_context('cp-2.cloudera...', 21..., 'tpch...')
and ran a long query and ctrl+c did not kill the query (I could see the query running on Impala query monitor even after seeing KeyboardInterrupt
):
>>> rs = conn.sql("select cntrycode, count(*) as numcust, sum(c_acctbal) as totacctbal from ( select substring(c_phone,1,2) as cntrycode, c_acctbal from tpch100g_parquet.customer where substring(c_phone,1,2) in ('13', '31', '23', '29', '30', '18', '17') and c_acctbal > ( select avg(c_acctbal) from tpch100g_parquet.customer where c_acctbal > 0.00 and substring(c_phone,1,2) in ('13', '31', '23', '29', '30', '18', '17')) and not exists ( select * from tpch100g_parquet.orders_scramble where o_custkey = c_custkey)) as custsale group by cntrycode order by cntrycode;")
^CTraceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/dyoon/Work/verdictdb-evan/pyverdict/pyverdict/verdictcontext.py", line 226, in sql
return self.sql_raw_result(query).to_df()
AttributeError: 'NoneType' object has no attribute 'to_df'
>>>
KeyboardInterrupt
>>>
KeyboardInterrupt
>>>
The query only terminated when I closed the shell itself with ctrl+d.
@ebatsell Please re-push this commit(s) after checking out this repository. By doing so, CircleCI should be triggered automatically.
When manually killing a pyverdict process with ctrl-c (when running a python script or in the python shell), verdictdb would previously continue running the query processes. I've tested for bypass statements and scramble tables in mysql and postgres and this solution seems to work, though note that this is hard to test consistently because one needs to start large queries in order for the problem to be noticeable.