CERT-Polska / karton

Distributed malware processing framework based on Python, Redis and S3.
https://karton-core.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
381 stars 45 forks source link

orjson.JSONDecodeError: number is infinity when parsed as double #224

Closed Antelox closed 1 year ago

Antelox commented 1 year ago

When parse_resources flag is passed to the unserialize method (for instance karton-system GC), it may raise exceptions from time to time resulting in having a GC not cleaning up the tasks/resources hence redis and S3 bucket will grow. Of course the exception depends on the JSON content which can be anything. An example of JSON which can reproduce the specific one reported here is an RSA parsed key:

>>> import orjson

>>> j = """{"cryptos": [{"key": {"e": 65537, "n": 16500472521988697010663112438705807640072764589479002554256260695717317064262484691290598356970646828426660349407212809681822203919126081272297252018228030950728477136113932493730960235450313211028445802933417736042246471737169968152424614291341808025585752848637795661794780420312612645575283548111399362423653839834760368929525863676430601132093478720051122016787902563729760403548823660511230280753799912283221065068155277355752466002679387284450151073598015621110653783630539129630982849849675891985711599794713831157549822527748863844615219682824485519877354977586980738215172053213147055330238573803265248619061}, "key_type": "RSA"}]}"""

>>> orjson.loads(j)

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

orjson.JSONDecodeError: number is infinity when parsed as double: line 1 column 40 (char 39)

The exact same input gets loaded correctly with the native json lib:

>>> import json

>>> json.loads(j)

{'cryptos': [{'key': {'e': 65537, 'n': 16500472521988697010663112438705807640072764589479002554256260695717317064262484691290598356970646828426660349407212809681822203919126081272297252018228030950728477136113932493730960235450313211028445802933417736042246471737169968152424614291341808025585752848637795661794780420312612645575283548111399362423653839834760368929525863676430601132093478720051122016787902563729760403548823660511230280753799912283221065068155277355752466002679387284450151073598015621110653783630539129630982849849675891985711599794713831157549822527748863844615219682824485519877354977586980738215172053213147055330238573803265248619061}, 'key_type': 'RSA'}]}

I'm thinking if makes sense to add a fallback to json.loads in the unserialize method in order to avoid exceptions in cases like the above one. I will create a PR in case this can sound like a fix.