VirusTotal / vt-py

The official Python 3 client library for VirusTotal
https://virustotal.github.io/vt-py/
Apache License 2.0
531 stars 121 forks source link

Object of type WhistleBlowerDict is not JSON serializable #176

Closed jxb5151 closed 7 months ago

jxb5151 commented 7 months ago

When attempting to update LiveHunt rules using the latest 0.18.0 version of vt-py I encountered the following error:

>>>    await client.patch_object_async(
  File "/home/test/.local/lib/python3.10/site-packages/vt/client.py", line 714, in patch_object_async
    response = await self.patch_async(path, *path_args, json_data=data)
  File "/home/test/.local/lib/python3.10/site-packages/vt/client.py", line 675, in patch_async
    await self._get_session().patch(
  File "/home/test/.local/lib/python3.10/site-packages/aiohttp/client.py", line 430, in _request
    data = payload.JsonPayload(json, dumps=self._json_serialize)
  File "/home/test/.local/lib/python3.10/site-packages/aiohttp/payload.py", line 396, in __init__
    dumps(value).encode(encoding),
  File "/usr/lib/python3.10/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type WhistleBlowerDict is not JSON serializable

The issue appears to have been introduced here: https://github.com/VirusTotal/vt-py/pull/175/files

More specifically, https://github.com/VirusTotal/vt-py/blob/944e58c4dd3780356a64c1796f752f2b86958a2e/vt/object.py#L25

Where collections.UserDict was added instead of dict as a fix to support a separate issue. Unfortunately, this appears to introduce an issue with supported objects/types the in the Json module.

More info: https://stackoverflow.com/questions/70762125/why-is-class-that-extends-dict-json-serializable-but-one-that-extends-collectio

The following has additional info on extending it, if that is a route you want to go down: https://stackoverflow.com/questions/57982946/how-to-register-implementation-of-abc-mutablemapping-as-a-dict-subclass

mgmacias95 commented 7 months ago

Hello @jxb5151,

Which python version are you using? Can you share a reproducer script? I tried the following and it worked correctly:

Python 3.11.7 (main, Dec  4 2023, 18:10:11) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import vt
>>> with vt.Client('xxxxx') as c:
...   obj = c.get_object('/intelligence/hunting_rulesets/12345')
...   c.patch_object('/intelligence/hunting_rulesets/12345', obj=obj)
... 
<vt.object.Object hunting_ruleset 12345>

Thanks!

plusvic commented 7 months ago

I'm having this same issue with Python 3.10.12. It can be reproduced by running examples/file_feed.py. It works fine with version 87ab6870ddb476c3cc9c854a89caaf8f1cf5374f

jxb5151 commented 7 months ago

Confirmed also using Python 3.10.6. I used the following to repro:

with vt.Client(api_key) as c:
    results = c.iterator('/intelligence/hunting_rulesets?filter=name:testing', limit=1)
    for obj in results:
        c.patch_object(f'/intelligence/hunting_rulesets/{obj.id}', obj=obj)

Produces:

Traceback (most recent call last):
  File "/home/test/dev/test.py", line 18, in <module>
    c.patch_object(f'/intelligence/hunting_rulesets/{obj.id}', obj=obj)
  File "/home/test/.local/lib/python3.10/site-packages/vt/client.py", line 703, in patch_object
    return make_sync(self.patch_object_async(path, *path_args, obj=obj))
  File "/home/test/.local/lib/python3.10/site-packages/vt/utils.py", line 27, in make_sync
    return event_loop.run_until_complete(future)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/test/.local/lib/python3.10/site-packages/vt/client.py", line 714, in patch_object_async
    response = await self.patch_async(path, *path_args, json_data=data)
  File "/home/test/.local/lib/python3.10/site-packages/vt/client.py", line 675, in patch_async
    await self._get_session().patch(
  File "/home/test/.local/lib/python3.10/site-packages/aiohttp/client.py", line 409, in _request
    data = payload.JsonPayload(json, dumps=self._json_serialize)
  File "/home/test/.local/lib/python3.10/site-packages/aiohttp/payload.py", line 398, in __init__
    dumps(value).encode(encoding),
  File "/usr/lib/python3.10/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type WhistleBlowerDict is not JSON serializable
mgmacias95 commented 7 months ago

Hello @plusvic and @jxb5151,

Thank you for your replies. I made a fix in #177.

Cheers