portals-project / portals

Portals is a framework for stateful serverless apps, unifying dataflow streaming with actors
https://www.portals-project.org/
Apache License 2.0
19 stars 2 forks source link

Serialization errors #154

Open jspenger opened 1 year ago

jspenger commented 1 year ago

In adding serialization we have encountered numerous errors with serialization. We should diagnose and deal with these errors. This is related to, and blocks the integration of, PR #130.

Problems: Currently, we are capturing all kinds of objects in our serialized data which are not serializable. Prominent examples include the Bank Account, in which we capture the AccountTask in the await body. This is then serialized, and causes an error. The essence of the problem with the AccountTask is that we call another method of the class itself from within the await block, capturing the this object itself, which shouldn't be serialziable. In this specific case, it should be possible and encouraged to instead write the application in a functional style with objects instead of classes. To deal with this we suggest the following steps.

Tasks:

To add an initial check, and avoid working with RocksDB for example, we can simply serialize all await blocks and all captured UDFs to see if they cause serialziation errors, as a first step.

To deal with the issues we should consider two things: 1) we can instead write the functions in more functional style (without classes, instead with objects/values); 2) integrate a check with the Spores3 library (which we are concurrently also working on integrating), that will check at compile-time for such issues.