Closed Depetrol closed 3 months ago
CI is failing because of #2376
I still cannot say that I like installing something into the user's system as a side effect of the preamble in a test file. I also think that requiring the serializer to be a standalone Python package that is installed separately is not a practical design from a usability perspective. However, I have made those points before and will not continue arguing for them. I cannot give this my approval, but if someone else does, I will not block it.
I agree that installing something into the user's system as a side effect of the preamble in a test file is not a perfect solution, but it doesn't require developers to install it manually in order to run the test, and makes the test self-contained.
However I would argue that installing a standalone Python package is more user friendly compared with using relative imports. There are several reasons:
out_port.set(custom_serialize_func(<object>))
..lf
code where it is written. The user would also have to specify a files
parameter every time the serializer is used to copy the serializer into the generated directory.@edwardalee sorry I re-requested a review but I hadn't noticed that you just provided feedback already
This PR enables a new feature for using arbitrary python package names that follows a specific API as the connection serializer in Python target federated execution.
Syntax:
This feature is backwards compatible. If
<serializer_name>
is any of[NATIVE, ROS2, PROTO]
, it will use these serialization methods instead of finding a python package.The custom Python package would have to export a
Serializer
class that has the following API:The
Serializer
class will be initialized at the first send and first receive on both ends of the connection. It is persisted in eachnetworksender
andnetworkreceiver
as a state variable. Python objects are serialized withserialize()
, sent over the socket and deserialized withdeserialize()
.This feature allows for user-defined custom serialization for any strategy. For example, the user could define a serialization method that enables fast inter-process communication with shared memory. Potentially ROS2 and PROTO serialization could be implemented with custom python packages too. It also supports different serialization methods for different connections.