JSONDecodeError

[michellemho]

Running into an issue when using cartoframes to read a table. Based on the traceback, I think it has to do with the carto python library SQL client and parsing a problematic json.

@andy-esch and I were able to get a dataframe back when we query with a limit (cc.query('select * from joined_full limit 1000). But once we try cc.query('select * from joined_full limit 100000), we get the error again. So this could indicate a problem with a few rows of the dataset or with some sort of query limit. Any ideas on what's happening here would be helpful!

If you need access to the joined_full dataset to help diagnose, just let me know.

Error:

---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
/anaconda3/lib/python3.6/site-packages/carto/sql.py in send(self, sql, parse_json, do_post, format, **request_args)
     83 
---> 84             return self.auth_client.get_response_data(resp, parse_json)
     85         except Exception as e:

/anaconda3/lib/python3.6/site-packages/pyrestcli/auth.py in get_response_data(self, response, parse_json)
     46             if parse_json:
---> 47                 return response.json()
     48             return response.content

/anaconda3/lib/python3.6/site-packages/requests/models.py in json(self, **kwargs)
    891                     pass
--> 892         return complexjson.loads(self.text, **kwargs)
    893 

/anaconda3/lib/python3.6/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    353             parse_constant is None and object_pairs_hook is None and not kw):
--> 354         return _default_decoder.decode(s)
    355     if cls is None:

/anaconda3/lib/python3.6/json/decoder.py in decode(self, s, _w)
    338         """
--> 339         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    340         end = _w(s, end).end()

/anaconda3/lib/python3.6/json/decoder.py in raw_decode(self, s, idx)
    354         try:
--> 355             obj, end = self.scan_once(s, idx)
    356         except StopIteration as err:

JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 50479105 (char 50479104)

During handling of the above exception, another exception occurred:

CartoException                            Traceback (most recent call last)
<ipython-input-25-6bdb9cec3095> in <module>()
----> 1 road_segments = cc.read('joined_full')

~/proj/cartoframes/cartoframes/context.py in read(self, table_name, limit, index, decode_geom, shared_user)
    200                 raise ValueError("`limit` parameter must an integer >= 0")
    201 
--> 202         return self.query(query, decode_geom=decode_geom)
    203 
    204     @utils.temp_ignore_warnings

~/proj/cartoframes/cartoframes/context.py in query(self, query, table_name, decode_geom)
    760                 query,
    761                 skipfields='the_geom_webmercator',
--> 762                 **DEFAULT_SQL_ARGS)
    763             if 'error' in select_res:
    764                 raise CartoException(str(select_res['error']))

/anaconda3/lib/python3.6/site-packages/carto/sql.py in send(self, sql, parse_json, do_post, format, **request_args)
     84             return self.auth_client.get_response_data(resp, parse_json)
     85         except Exception as e:
---> 86             raise CartoException(e)
     87 
     88 

CartoException: Expecting property name enclosed in double quotes: line 1 column 50479105 (char 50479104)

[andy-esch]

We can reproduce in the python sdk with

sql_client = SQLClient(...)
sql_client.send('select * from joined_full limit 100000')

[ethervoid]

This is related to issue https://github.com/CartoDB/carto-python/issues/89

[andy-esch]

This should be fixed now: https://github.com/CartoDB/support/issues/1688