qubole / qds-sdk-py

Python SDK for accessing Qubole Data Service
https://qubole.com
Apache License 2.0
51 stars 127 forks source link

utf-8 errors with qds.py when retrieving results #164

Open amelio-vazquez-reina opened 8 years ago

amelio-vazquez-reina commented 8 years ago

I just run into this problem:

$ qds.py --token=$QUBOLE_KEY_AVAZQUEZ hivecmd getresult 28319250

Traceback (most recent call last):
  File "/Users/amelio/anaconda/envs/py35/bin/qds.py", line 603, in <module>
    sys.exit(main())
  File "/Users/amelio/anaconda/envs/py35/bin/qds.py", line 556, in main
    return cmdmain(a0, args)
  File "/Users/amelio/anaconda/envs/py35/bin/qds.py", line 194, in cmdmain
    return globals()[action + "action"](cmdclass, args)
  File "/Users/amelio/anaconda/envs/py35/bin/qds.py", line 160, in getresultaction
    return _getresult(cmdclass, cmd)
  File "/Users/amelio/anaconda/envs/py35/bin/qds.py", line 118, in _getresult
    cmd.get_results(sys.stdout, delim='\t')
  File "/Users/amelio/anaconda/envs/py35/lib/python3.5/site-packages/qds_sdk/commands.py", line 207, in get_results
    skip_data_avail_check=isinstance(self, PrestoCommand))
  File "/Users/amelio/anaconda/envs/py35/lib/python3.5/site-packages/qds_sdk/commands.py", line 1286, in _download_to_local
    _read_iteratively(one_path, fp, delim=delim)
  File "/Users/amelio/anaconda/envs/py35/lib/python3.5/site-packages/qds_sdk/commands.py", line 1177, in _read_iteratively
    fp.buffer.write(data.decode('utf-8').replace(chr(1), delim).encode('utf8'))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe6 in position 8191: unexpected end of data
Fatal Python error: GC object already tracked

Current thread 0x00007fff7c065000 (most recent call first):
amelio-vazquez-reina commented 8 years ago

I'm having this problem even after updating to the latest qds-sdk:

$ pip freeze | grep qds
qds-sdk==1.9.3
amelio-vazquez-reina commented 8 years ago

FYI: I don't have this problem in Python 2.7.x. I only have it in Python 3.5.x.