Possible adaptation of JSONSerializableClass

[dlohmeier]

There is something I wanted to document with respect to the JSONSerializableClass. Currently the way it is restored from json is that a dictionary is read from the json file and then handed over to the from_dict method. This dictionary has to contain the '_init' parameters saved by calling update_initialized and the '_state' parameters just extracted from the __dict__ right before saving to json. The object is then restored by calling the constructor with the '_init' parameters and updating the __dict__ with the '_state' parameters.

As far as I found out in a small research it should be possible to create an object of the inheriting class with the __new__() operator and then just updating the __dict__ with the '_state' parameters. I see two advantages in this:

1. It is not necessary to always call update_initialized which makes it simpler for anyone who wants to create his own new class.
2. If an object of JSONSerializable contains another object of JSONSerializable, this one is contained in both the '_init' and the '_state' parameters. Therefore it will be initialized twice and once just thrown away once the __dict__ is updated.

What we probably still need is a workaround for monkey-patched methods as they are not contained in the __dict__ of the object. But I think this is also not yet considered in the current version, as far as I know.

Unfortunately, this all doesn't seem to be very trivial, so it would take some time that I don't have at the moment. I will see if I can look into this soon, but if someone else will work on this class, maybe that person can have a look at my proposal and implement it as well or get in contact with me.

[dlohmeier]

One additional aspect is how to save and initialize monkey patched methods. The following would be possible:

import marshal, base64

def func1(x):
    return x + 1

a = lambda x: x + 2

f1_string = base64.b64encode(marshal.dumps(func1.__code__))
f1_copy = types.FunctionType(marshal.loads(base64.b64decode(f1_string)), globals())

a_string = base64.b64encode(marshal.dumps(a.__code__))
a_copy = types.LambdaType(marshal.loads(base64.b64decode(a_string)), globals())

print(func1(1000), f1_copy(1000), a(1000), a_copy(1000))

The main problems with this implementation are:

• It is vulnerable to corrupted files, i.e. if you load a net with a virus written into one of the functions as bytecode, it might load a virus to your system. Maybe there are ways to avoid that??
• It will probably lead to long printouts of not readable byte strings.

Alternatively, if functions are defined within a certain namespace, it should be possible to load them from this namespace, as it is done with basically all parts of a pandapower network. It should be worth discussing which way to go (maybe both ways should be implemented, so that only anonymous functions directly monkey patched into the class will be saved as byte strings, the rest will be tried to retrieve from the namespace)...

[rbolgaryn]

I have implemented the approach with new on my branch. There are some points to think about:

• It is possible to simplify pp_hook quite a bit, as well as JSONSerializableObject
• update_initialized() is no longer necessary
• restore_jsoned_objects can be avoided, for that an empty net can be initialized in PPJSONDecoder and given to pp_hook using partial(pp_hook(net=net)). Then, the empty net is written to the jsoned objects via obj.dict.update({'net':net, **d})). At a later point, the net itself is filled with the respective elements, but the pointer to it is the same. Therefore, json.loads(s, cls=PPJSONDecoder) already returns the net with all the jsoned objects already unpacked.
• Caveat: it is implemented for using DataFrame.to_dict() rather than to_json (as before) because otherwise, it would be trickier. I will look into the performance and file size implications.
• It should be possible to implement it as it was before, with DataFrame.to_json, but I need to implement an additional deserialization at a correct point.
• Tests don't work right now due to ConstControl having monkey patching in init. We could either require that no monkey-patching is used in such classes or add the feature to JSONSerializableObject to save monkey-patched functions. I think it is possible, that a function module and name are added in this case and then the function is imported and assigned to the object. Apart from that, other tests are working.

At the moment, further todos are:

• to_dict vs to_json for DataFrame, implement for to_json
• Add the possibility to serialize monkey-patched functions or introduce a requirement to avoid monkey-patching. I suggest choosing the first option.
• Test the approach of initializing net in PPJSONDecoder with corner cases, e.g. having several net objects in 1 json file etc. to be sure it is a stable solution.

[rbolgaryn]

Tests are running through now -> #559

[rbolgaryn]

• to_json implemented for the new approach
• monkey-patched functions can be serialized, although class methods cannot
• update_initialized is removed
• restore_jsoned_objects is removed
• ConstControl has been changed to avoid the requirement for serialization for class methods: monkey-patching using functions is supported, monkey-patching using class methods is not supported and should be avoided