hazelcast / hazelcast

Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
https://www.hazelcast.com
Other
6.08k stars 1.83k forks source link

Portable serialization with serializer outside domain class #13041

Open TomaszGaweda opened 6 years ago

TomaszGaweda commented 6 years ago

Hi,

this is a proposal of new way of defining Portable serialization.

Current approach is that domain model class must implement Portable, so it must implement write and read methods. It's quite messy for large objects.

New way is to create new class with serialization logic for each domain model class. It's like in POF serialization - there you can have class A and APofSerializer.

BR

mmedenjak commented 6 years ago

Hi @TomaszGaweda !

Thanks for the suggestion. By POF you mean this?

https://docs.oracle.com/cd/E24290_01/coh.371/e23131/usepof.htm#CHDDFHGA https://docs.oracle.com/middleware/1213/coherence/java-reference/com/tangosol/io/pof/PofSerializer.html

TomaszGaweda commented 6 years ago

Hi @mmedenjak,

Thanks for your comment :) Yeah, I mean Coherence POF. PofSerializer outside entity often is a cleaner approach, especially if entities are quite large, something similar in Hazelcast may be also helpful

ph33rtehgd commented 6 years ago

I'd like to second this request. This is useful in a number of use cases.

  1. You need to serialize an object to a Portable format that isn't under your control (like third party libraries).

  2. You want to keep your domain objects decoupled from a specific caching solution.

In my particular use case I have to support multiple caching implementations (Coherence, Ignite, Hazelcast, etc.) while using platform independent formats (our UI is .NET) without putting any hard dependencies in our core model on any given solution. Currently this doesn't seem possible with Hazelcast as it requires the implementation of the Portable interface. An external class that can be assigned the responsibility of doing the Portable serialization would fix this problem.

mmedenjak commented 3 years ago

@sancar is something like this planned or do you have some idea how to provide something like this?

sancar commented 3 years ago

It is not something hard to add. The only problem is PortableReader/PortableWriter methods need new methods for read/writePortable where they don't expect a Portable interface.

That being said, we probably address this in the new format rather than Portable.

mmedenjak commented 3 years ago

I was referring to that - are we planning to do something like that (external schemas) in the new format?

sancar commented 3 years ago

Yes, it is the plan for the new format. Note that The API is still subject the change.

Instead of implementing an interface, user will be able to register a serializer for a class as follows:

serializationConfig.register(Employee.class, "employee", new CompactSerializer<Employee>() {
               @Override
               public Employee read(CompactReader in) throws IOException {
                   String name = in.readUTF("name");
                   int age = in.readInt("age");
                   return new Employee(name, age);
               }

               @Override
               public void write(CompactWriter out, Employee object) throws IOException {
                   out.writeUTF("name", object.getName());
                   out.writeInt("age", object.getAge());
               }
           });

Actually, the plan is not to get this config from the user at all when we can. Only the object is complicated enough, the user will have to fallback to this API.