moj-analytical-services / pydbtools

Python version of dbtools
https://moj-analytical-services.github.io/pydbtools/
11 stars 2 forks source link

DECIMAL presto type not supported #3

Closed isichei closed 3 years ago

isichei commented 5 years ago
import pydbtools as pydb

sql = """
SELECT CAST(0 AS DECIMAL(1,1)) as test
"""
out = pydb.read_sql(sql)

=== Error ===

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-1-f290daede81f> in <module>
      4 SELECT CAST(0 AS DECIMAL(1,1)) as test
      5 """
----> 6 out = pydb.read_sql(sql)

~/.local/lib/python3.6/site-packages/pydbtools/read_sql.py in read_sql(sql_query, timeout, *args, **kwargs)
     17 
     18     # Read in the SQL query
---> 19     dtype, parse_dates = _pd_dtype_dict_from_metadata(response['meta'])
     20     s3_path = response['s3_path'].replace('s3://', 's3a://')
     21     df = pd.read_csv(s3_path, dtype=dtype, parse_dates=parse_dates, *args, **kwargs)

~/.local/lib/python3.6/site-packages/pydbtools/utils.py in _pd_dtype_dict_from_metadata(athena_meta)
     57     for c in athena_meta:
     58         colname = c["name"]
---> 59         coltype = _athena_meta_conversions[c["type"]]['pandas']
     60         dtype[colname] = np.typeDict[coltype]
     61         if c["type"] in ["date", "timestamp"]:

KeyError: 'decimal'