search

plone / guillotina

Python AsyncIO data API to manage billions of resources
https://guillotina.readthedocs.io/en/latest/
Other
187 stars 51 forks source link

Store JSON objects instead of pickle objects #1051

Open masipcat opened 3 years ago

masipcat commented 3 years ago

https://github.com/plone/guillotina/issues/1026

I'll rebase this branch with master when master is G7

masipcat commented 3 years ago

Almost everything works. Only need to fix tests that manipulate files

masipcat commented 3 years ago

All tests green! But there are some things to consider/discuss:

  1. I think the current implementation is vulnerable. If someone sends a dict like {"__class__": ...} on a JSONField, we'll resolve the dotted name and create an instance of the __class__ when reading the object from db. The first thing that comes to my mind to mitigate this is appending a secret key like this __class__y2o930874h5f8764598. This requires each guillotina installation to generate this key and put in in the config. What do you think? Any other idea on how to solve it?

  2. orjson is strict and doesn't serialize keys that aren't strings and I found some objects like Blob and BucketList that have dicts with numeric keys. I solved these cases doing a pickle of the object :P This is something that the developer of the application can solve for their own types, but needs to implement custom IStorageSerializer/Deserializer and might be a little inconvenient.