JSON serialization failing as of today on a handful of different python sdk methods

[jsnb-devoted]

Did something change in the sdk or the API in just the last day? We are on looker-sdk==23.16.0 and the 4.0 api. As of today we started receiving type errors that objects are not JSON serializable. Is there maybe an easier way to skip the deserialization step that puts these in the SDK models? Do I have to traverse all of the keys/items of the model data and turn them into dicts in order to get the raw data?

import json

logger = logging.Logger(__name__)

import looker_sdk

sdk = looker_sdk.init40()
sdk_method_name = "all_boards"
sdk_method = getattr(sdk, sdk_method_name)

try:
    retrieved_models = sdk_method()
except json.decoder.JSONDecodeError as e:
    logger.error(f"Failed to retrieve models: {e}")

flattened_base_data = []
for retrieved_model in retrieved_models:
    flattened_data = {}
    for key, value in retrieved_model.items():
        flattened_data[key] = value
    flattened_base_data.append(flattened_data)

for row in flattened_base_data:
    json.dumps(row)

    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type BoardSection is not JSON serializable

I even tried using the serialize function with the 4.0 converter directly but that got a serialization error datetime:

    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type DateTime is not JSON serializable

[mikenaux]

I am having the same problem, using looker-sdk==22.16.0

[anyapriya]

My coworker ran into a similar issue running looker-sdk==23.16.0 on python 3.11.5 (error output below). I'm running the same script, with the same looker-sdk version (23.16.0), but on python 3.9.13 and am NOT running into the same issue.

  File "/Users/bobadams/looker_deployer/deploy_embed_dashboards.py", line 233, in create_or_update_dashboard
    target_sdk.create_scheduled_plan(body = schedule_model)
  File "/opt/homebrew/anaconda3/lib/python3.11/site-packages/looker_sdk/sdk/api40/methods.py", line 10429, in create_scheduled_plan
    self.post(
  File "/opt/homebrew/anaconda3/lib/python3.11/site-packages/looker_sdk/rtl/api_methods.py", line 171, in post
    serialized = self._get_serialized(body)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/lib/python3.11/site-packages/looker_sdk/rtl/api_methods.py", line 156, in _get_serialized
    serialized = self.serialize(api_model=body)  # type: ignore
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/lib/python3.11/site-packages/looker_sdk/rtl/serialize.py", line 81, in serialize
    return json.dumps(data).encode("utf-8")  # type: ignore
           ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/lib/python3.11/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/lib/python3.11/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/lib/python3.11/json/encoder.py", line 258, in iterencode
    return _iterencode(o, 0)
           ^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/lib/python3.11/json/encoder.py", line 180, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type ScheduledPlanDestination is not JSON serializable

[michaelchu]

There seems to be a breaking change from the cattrs package(a dependancy of looker-sdk) for versions after 23.1.2. Downgrading it seems to work for us, this is with looker-sdk==23.16.0 and python 3.9.15

[ldnicolasmay]

We're getting a similar error with Python 3.11.1, looker-sdk 23.16.0, 4.0 API.

Offending code:

def disable_users(sdk: mtds.Looker40SDK, user_ids: List[str]) -> None:
    """
    Disable a list of users
    :param sdk: Looker API SDK object
    :param user_ids: List of user IDs
    :return: None
    """
    for user_id in user_ids:
        user = sdk.user(user_id)
        assert isinstance(user, mls.User)
        assert not user.is_disabled
        user.is_disabled = True
        sdk.update_user(user_id=user.id, body=user)
        assert user.is_disable

Stacktrace:

Traceback (most recent call last):
  File "/workdir/scripts/user_management/user_audit.py", line 201, in <module>
    main(
  File "/workdir/scripts/user_management/user_audit.py", line 141, in main
    user_audit_utils.disable_users(sdk, inactive_user_ids_to_disable)
  File "/workdir/scripts/user_management/user_audit_utils.py", line 240, in disable_users
    sdk.update_user(user_id=user.id, body=user)
  File "/usr/local/lib/python3.11/site-packages/looker_sdk/sdk/api40/methods.py", line 11423, in update_user
    self.patch(
  File "/usr/local/lib/python3.11/site-packages/looker_sdk/rtl/api_methods.py", line 192, in patch
    serialized = self._get_serialized(body)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/looker_sdk/rtl/api_methods.py", line 156, in _get_serialized
    serialized = self.serialize(api_model=body)  # type: ignore
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/looker_sdk/rtl/serialize.py", line 81, in serialize
    return json.dumps(data).encode("utf-8")  # type: ignore
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/encoder.py", line 258, in iterencode
    return _iterencode(o, 0)
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/encoder.py", line 180, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type CredentialsGoogle is not JSON serializable

[vj-devs]

Downgrading the cattrs package resolved this issue on Python 3.10.5 with the latest Looker SDK. It might also work with other versions

pip install cattrs==22.2.0

[jicuss]

awesome guys, this worked for me pip install cattrs==23.1.2 --upgrade --force

[oscar-garza-66]

pip install cattrs==23.1.2 --upgrade --force also solved my issue

[jeremytchang]

Closed as fixed.