getting token expired issues in dremio simple client

[vikashsingh009]

• Dremio client version:
• Dremio version:
• Python version:
• Operating System:

Description

I have written a python api using dremio simple cleint and running on ubuntu VM.it was working fine from past 4-5 days but sudenly server responding with Error

What I Did

here  is  python  api  to ftech  dremion dataset  details
import requests
from dremio_client import init
from dremio_client.flight import query
import json
from concurrent.futures import as_completed
from flask_cors import CORS
import pandas as pd
# Load an empty map
from flask import Flask, request, jsonify
app = Flask(__name__)
app.config["DEBUG"] = True
CORS(app)
from waitress import serve
import urllib3

requests.packages.urllib3.disable_warnings()
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL'
try:
    requests.packages.urllib3.contrib.pyopenssl.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL'
except AttributeError:
    # no pyopenssl support used / needed / available
    pass
MapConfig = ''
@app.route('/api/', methods=['GET','POST'])
def api_get():
    if request.method == 'GET':
        url = 'https://testdremio.test.com/apiv2/login'
        headers  = {"content-type": "application/json","Connection": "keep-alive"}
        data = json.dumps({"userName": "*****", "password": "*****"})
        response = requests.post(url, data=data,headers = headers,verify=False )
        print(response.json())
        url = 'https://testdremio.test.com/apiv2/space/Test_Space/folder/Datasets'
        authkey='_dremio'+response.json()['token']
        print(authkey)
        headers  = {"Accept": "application/json","Authorization": authkey}
        print(headers)
        response = requests.get(url, headers = headers,verify=False )
        print(response.json())
        return response.json()
    elif request.method == 'POST':
        data=None
        global MapConfig
        print('post app')
        requestPayLoad = json.loads(json.dumps(request.json))
        print(requestPayLoad)
        if requestPayLoad['Type'] == 'UpdateMapConfig':
            MapConfig=requestPayLoad['Value']
            data='SUCCESS'
        elif requestPayLoad['Type'] == 'ReadMapConfig':
            data=MapConfig
        else:
            query=requestPayLoad['Value']
            client = init(simple_client=True)
            results = client.query(query)
            data=pd.DataFrame(results)
            data=data.to_json()
        return  data;

    else:
        return 'not OK'
serve(app,host='0.0.0.0', port=5001)

and  i  am  getting  below  error

json.decoder.JSONDecodeError: Extra data: line 1 column 124 (char 123)
ERROR:waitress:Exception while serving /api/
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/waitress/channel.py", line 350, in service
    task.service()
  File "/usr/local/lib/python3.6/dist-packages/waitress/task.py", line 171, in service
    self.execute()
  File "/usr/local/lib/python3.6/dist-packages/waitress/task.py", line 441, in execute
    app_iter = self.channel.server.application(environ, start_response)
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 2464, in _call_
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 2450, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python3.6/dist-packages/flask_cors/extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1867, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/dist-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.6/dist-packages/flask_cors/extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/dist-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "server.py", line 29, in api_get
    client = init(simple_client=True)
  File "/usr/local/lib/python3.6/dist-packages/dremio_client/__init__.py", line 67, in init
    return _connect(config, simple_client)
  File "/usr/local/lib/python3.6/dist-packages/dremio_client/__init__.py", line 72, in _connect
    return SimpleClient(config)
  File "/usr/local/lib/python3.6/dist-packages/dremio_client/dremio_simple_client.py", line 82, in _init_
    self._token = auth(self._base_url, config)
  File "/usr/local/lib/python3.6/dist-packages/dremio_client/auth/__init__.py", line 41, in auth
    return _existing_token(config_dict)
  File "/usr/local/lib/python3.6/dist-packages/dremio_client/auth/__init__.py", line 73, in _existing_token

teporary solution if i delete auth.json file it will start responding .how to solve this token issues so that i dont have to maual delete auth.json

[vikashsingh009]

@rymurr any comments on above issues

[rymurr]

Hey @vikashsingh009 really sorry for the delay! Been v busy. Will give this a look this week for sure!

[vikashsingh009]

i guess here is the issues https://github.com/rymurr/dremio_client/blob/master/dremio_client/auth/__init__.py#L73

File "/usr/local/lib/python3.6/dist-packages/dremio_client/auth/init.py", line 73, in _existing_token authfile = json.load(f) File "/usr/lib/python3.6/json/init.py", line 299, in load parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw) File "/usr/lib/python3.6/json/init.py", line 354, in loads return _default_decoder.decode(s) File "/usr/lib/python3.6/json/decoder.py", line 342, in decode raise JSONDecodeError("Extra data", s, end)

[vikashsingh009]

For refrance i see that auth.json having not valid json ,here is the file,i am not sure why extra braces appending

[rymurr]

hey @vikashsingh009 I just tried this on the most recent version of dremio_client and python 3.8. My resulting auth.json didn't have the extra } that you are seeing. Any chance you are using an old or previously edited version of dremio_client?

[vikashsingh009]

@rymurr i also can't able to reproduce that issues ,as for some time its a valid json ,but if api continues open too may sessions in that case token getting extra braces