Sinaptik-AI / pandas-ai

Chat with your database (SQL, CSV, pandas, polars, mongodb, noSQL, etc). PandasAI makes data analysis conversational using LLMs (GPT 3.5 / 4, Anthropic, VertexAI) and RAG.
https://pandas-ai.com
Other
11.67k stars 1.07k forks source link

add: json lib to the the environment #1228

Closed shyoji closed 1 week ago

shyoji commented 1 week ago
gventuri commented 1 week ago

@shyoji thanks for the PR! Could you share some more context about what's the purpose of these changes?

shyoji commented 1 week ago

@shyoji thanks for the PR! Could you share some more context about what's the purpose of these changes?

When receiving a code response from an LLM model that includes a stringified JSON object, such as {'type': 'string', 'value': '{'message': 'The total current term balance is 34587053.03.', 'charts': [], 'tables': []}'}, converting the value to a dictionary format might throw an error if the context environment lacks the JSON library. This issue can be resolved by ensuring that the json library is imported and using json.loads() to properly parse the JSON string into a dictionary. This approach ensures the value field is correctly converted, preventing errors related to JSON decoding and maintaining the integrity of the data handling process.

shyoji commented 1 week ago

@gventuri Can u please check this PR?

shyoji commented 1 week ago

So If I am getting this kind of code response


    promoters = df[df['LikelihoodToRecommend'] >= 9].shape[0]
    detractors = df[df['LikelihoodToRecommend'] <= 6].shape[0]
    nps = (promoters - detractors) / df.shape[0] * 100
    return nps

average_nps = calculate_average_nps(dfs[0])
result = {'type': 'string', 'value': {'message': 'The average NPS is ' + str(average_nps) + '.', 'charts': [], 'tables': []}}
request_chart = True
if request_chart:
    result['value']['charts'] = [{'chart_type': 'BAR_CHART', 'data': [{'label': 'NPS', 'value': float(average_nps)}]}]
print(json.dumps(result))```

Environment :- {'pd': <module 'pandasai.pandas' from '/Users/shyojimeena/Desktop/workspace/skizaa-chatbot/shy2-venv/lib/python3.11/site-packages/pandasai/pandas/__init__.py'>, 'plt': <module 'matplotlib.pyplot' from '/Users/shyojimeena/Desktop/workspace/skizaa-chatbot/shy2-venv/lib/python3.11/site-packages/matplotlib/pyplot.py'>, 'np': <module 'numpy' from '/Users/shyojimeena/Desktop/workspace/skizaa-chatbot/shy2-venv/lib/python3.11/site-packages/numpy/__init__.py'>, '__builtins__': {'abs': <built-in function abs>, 'all': <built-in function all>, 'any': <built-in function any>, 'ascii': <built-in function ascii>, 'bin': <built-in function bin>, 'bool': <class 'bool'>, 'bytearray': <class 'bytearray'>, 'bytes': <class 'bytes'>, 'callable': <built-in function callable>, 'chr': <built-in function chr>, 'classmethod': <class 'classmethod'>, 'complex': <class 'complex'>, 'delattr': <built-in function delattr>, 'dict': <class 'dict'>, 'dir': <built-in function dir>, 'divmod': <built-in function divmod>, 'enumerate': <class 'enumerate'>, 'filter': <class 'filter'>, 'float': <class 'float'>, 'format': <built-in function format>, 'frozenset': <class 'frozenset'>, 'getattr': <built-in function getattr>, 'hasattr': <built-in function hasattr>, 'hash': <built-in function hash>, 'help': Type help() for interactive help, or help(object) for help about object., 'hex': <built-in function hex>, 'id': <built-in function id>, 'int': <class 'int'>, 'isinstance': <built-in function isinstance>, 'issubclass': <built-in function issubclass>, 'iter': <built-in function iter>, 'len': <built-in function len>, 'list': <class 'list'>, 'locals': <built-in function locals>, 'map': <class 'map'>, 'max': <built-in function max>, 'memoryview': <class 'memoryview'>, 'min': <built-in function min>, 'next': <built-in function next>, 'object': <class 'object'>, 'oct': <built-in function oct>, 'ord': <built-in function ord>, 'pow': <built-in function pow>, 'print': <built-in function print>, 'property': <class 'property'>, 'range': <class 'range'>, 'repr': <built-in function repr>, 'reversed': <class 'reversed'>, 'round': <built-in function round>, 'set': <class 'set'>, 'setattr': <built-in function setattr>, 'slice': <class 'slice'>, 'sorted': <built-in function sorted>, 'staticmethod': <class 'staticmethod'>, 'str': <class 'str'>, 'sum': <built-in function sum>, 'super': <class 'super'>, 'tuple': <class 'tuple'>, 'type': <class 'type'>, 'vars': <built-in function vars>, 'zip': <class 'zip'>, '__build_class__': <built-in function __build_class__>, '__name__': '__main__'},
Then I am getting this response. 

NameError: name 'json' is not defined
Exception in APILogger: <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot POST /api/log/add</pre>
</body>
</html>

Unfortunately, I was not able to answer your question, because of the following error:

name 'json' is not defined
gventuri commented 1 week ago

@shyoji I think the problem stands in the LLM generation, as it's not expected to return a json output, but should rather return the string in that case. Which llm are you using?

shyoji commented 1 week ago

@gventuri We are using llama

gventuri commented 1 week ago

@shyoji I think the problem is in the output being generated more than the lack of the json. I suggest to either update the prompt slightly or as an alternative pass more context to the message about how to return the output and configure the result variable.

shyoji commented 1 week ago

Okay thanks @gventuri