The information traverse the CLR-JVM boundary many times reducing the speed

[masesdevelopers]

          The information traverse the CLR-JVM boundary many times reducing the speed.

The local serializers stub invokes remote serializers: each time there is a conversion the JVM is impacted like in https://github.com/masesgroup/KNet/blob/721fe074212d4e221630853368beaab6a11bf120/src/net/KNet/Specific/Serialization/KNetSerialization.cs#L240 or https://github.com/masesgroup/KNet/blob/721fe074212d4e221630853368beaab6a11bf120/src/net/KNet/Specific/Serialization/KNetSerialization.cs#L347

Each conversion moves data between CLR and JVM:

• data (bool, int, double, etc) is sent to JVM
• the JVM returns a converted byte array
• the byte array is returned to the caller that insert it within e.g. a ProducerRecord
• ProducerRecord send back again the array to the JVM

The same happens in opposite with ConsumerRecord:

• JVM receives a ConsumerRecord
• if one data is needed, e.g. the key, the byte array is requested to the JVM which send it back to the CLR
• then CLR sent the byte array back again to the JVM
• finally the JVM returns the converted type (bool, int, double, etc)

Originally posted by @masesdevelopers in https://github.com/masesgroup/KNet/issues/53#issuecomment-1984765101

[masesdevelopers]

It is possible to add a property, named UseKafkaClassForSupportedTypes with default value set to false, to decide which kind of serializer use:

• false: the serializer will be the .NET one, e.g. int will be converted with BitConverter.GetBytes into byte array and BitConverter.ToInt32 from byte array to int:
  • Pro: the serialization happens in place and only the final result is transferred
  • Cons: the resulting data cannot be used from an application based on Apache Kafka serializer; the following are the snippet of IntegerSerializer and BitConverter.GetBytes:

Apache Kafka	CLR (source from GitHub)
```java public class IntegerSerializer implements Serializer { public byte[] serialize(String topic, Integer data) { if (data == null) return null; return new byte[] { (byte) (data >>> 24), (byte) (data >>> 16), (byte) (data >>> 8), data.byteValue() }; } } ```	```c# public static byte[] GetBytes(int value) { byte[] bytes = new byte[sizeof(int)]; Unsafe.As(ref bytes[0]) = value; return bytes; } ```

• true: the current serializer will be used, e.g. int will be converted with IntegerSerializer.Serialize into byte array and IntegerDeserializer.Deserialize from byte array to int:
  • Pro: there is full compatibility is serialization
  • Cons: current behavior is preserved like the CLR-JVM boundary traversing

Meanwhile on the property UseHeaders will be added the public setter to give to users of subclasses of SerDes, e.g. JSonSerDes, to remove the constraint imposed by the code in case of needs.