search

masesgroup / KNet

KNet is a comprehensive .NET suite for Apache Kafka™ providing all features: Producer, Consumer, Admin, Streams, Connect, backends (ZooKeeper and Kafka).
https://knet.masesgroup.com/
Apache License 2.0
36 stars 6 forks source link

.NET 8 raises `System.AccessViolationException` when `Org.Apache.Kafka.Streams.KeyValue<K, V>` is retrieved from `Org.Apache.Kafka.Streams.State.KeyValueIterator<K, V>` #345

Closed masesdevelopers closed 7 months ago

masesdevelopers commented 7 months ago

Describe the bug Under .NET 8 environment Org.Apache.Kafka.Streams.State.KeyValueIterator<K, V> many times raises System.AccessViolationException when Org.Apache.Kafka.Streams.KeyValue<K, V> is retrieved

To Reproduce See masesgroup/KEFCore#194

Expected behavior Since in .NET 6 and .NET 7 the issue is not raised, the expectation is the same for .NET 8

Screenshots N/A

Desktop (please complete the following information):

Additional context See masesgroup/KEFCore#194

masesdevelopers commented 7 months ago

Seems that the .NET Garbage Collector retires the Org.Apache.Kafka.Streams.KeyValue<K, V> object, releasing the underlying JVM object; this implies that the pointer stored is no more valid and the access to the data raises the exception. A possible workaround can be to instruct the .NET Garbage Collector to stop its execution around the critical region using GC.TryStartNoGCRegion ending it with GC.EndNoGCRegion:

#if NET8_0
bool startedGCRegion = false;
startedGCRegion = GC.TryStartNoGCRegion(1024, false);
#endif

// critical region

#if NET8_0
if (startedGCRegion && GCSettings.LatencyMode == GCLatencyMode.NoGCRegion) GC.EndNoGCRegion();
#endif

The memory, in the previous example is 1024, can be changed based on the application using the Org.Apache.Kafka.Streams.State.KeyValueIterator<K, V>, the second parameter is set to false to avoid an usefulness full GC.