SNOW-584971: fetch_pandas_all returns JSON columns as string not dict

I believe this is similar to several other issues and PRs:

What version of Python are you using?

python 3.9.7

What operating system and processor architecture are you using?

Linux-5.10.47-linuxkit-x86_64-with-glibc2.31

What are the component versions in the environment (pip freeze)? Built wheel using: ci/build_docker.sh which generates snowflake_connector_python-2.7.6-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
What did you do?

Create a DB with json data. I check that it is being stored in a JSON friendly format because i can query the data using : dotted access

cursor = ctx.cursor().execute(f"""
    SELECT 
      *
    FROM MYDATASET.PUBLIC.MYTABLE_VARIANT
    WHERE DATA:jsonPayload:bytes_sent > 3500
    LIMIT 2
    """)
result = cursor.fetch_pandas_all()
print(type(result['DATA'].iloc[0]), result['DATA'].iloc[0])

The output shows that it is <class 'str'>.

<class 'str'> {
  "insertId": "sb2haif1xnn2k",
  "jsonPayload": {
    "bytes_sent": "0",
    "connection": {
      "dest_ip": "2xx.85.147.95",
      "dest_port": "443",
      "protocol": "6",
      "src_ip": "10.128.0.xx",
      "src_port": "48516"
    },

What did you expect to see?

I expected that the dataframe column for DATA would automatically convert to JSON, in this case: <class 'dict'>.

If I use the Snowflake UI, I can copy the DATA column and it pastes as JSON. It also displays as JSON.

Can you set logging to DEBUG and collect the logs?

I don't think this is necessary since this is less a bug than a feature request. However, i marked it as a bug because I think it is a bug with respected to everybody's expectation of how it would work, and related to the many issues raised over fetch_pandas_all() not respecting schema.

snowflakedb / snowflake-connector-python

SNOW-584971: fetch_pandas_all returns JSON columns as string not dict #1115