ClassCastException is thrown when org.apache.river.jeri.tcp.useNIO=true option is used

[bharathkumara]

I have built the latest version of JGDMS locally and am able to run the lookup service without any issues. But when we use LookupDiscovery to discover lookup services, ClassCastException is thrown. I have used the System property org.apache.river.jeri.tcp.useNIO=true to use NIO with TCPServerEndpoint.

The following exception is thrown while discovering registrars.

java.lang.ClassCastException: class [Ljava.nio.Buffer; cannot be cast to class [Ljava.nio.ByteBuffer; ([Ljava.nio.Buffer; and [Ljava.nio.ByteBuffer; are in module java.base of loader 'bootstrap') at org.apache.river.jeri.internal.mux.SocketChannelConnectionIO.asyncSend(SocketChannelConnectionIO.java:159) at org.apache.river.jeri.internal.mux.Mux.asyncSendData(Mux.java:619) at org.apache.river.jeri.internal.mux.MuxOutputStream.writeBuffer(MuxOutputStream.java:237) at org.apache.river.jeri.internal.mux.MuxOutputStream.close(MuxOutputStream.java:120) at java.base/java.io.ObjectOutputStream$BlockDataOutputStream.close(ObjectOutputStream.java:1834) at java.base/java.io.ObjectOutputStream.close(ObjectOutputStream.java:745) at org.apache.river.api.io.AtomicMarshalOutputStream.close(AtomicMarshalOutputStream.java:129) at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:861) at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethod(BasicInvocationHandler.java:728) at net.jini.jeri.BasicInvocationHandler.invoke(BasicInvocationHandler.java:597) at net.jini.jeri.AtomicInvocationHandler.invoke(AtomicInvocationHandler.java:315) at com.sun.proxy.$Proxy11.getLocator(Unknown Source) at org.apache.river.reggie.proxy.RegistrarProxy.getLocator(RegistrarProxy.java:

Is this issue related to it? https://github.com/pfirmstone/JGDMS/issues/117 I have used JDK 11 to build and run it.

I tried to run it without useNIO option and UnmarshalException is thrown.

java.rmi.UnmarshalException: exception unmarshalling response; nested exception is: java.io.IOException: request aborted by remote endpoint at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:929) at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethod(BasicInvocationHandler.java:728) at net.jini.jeri.BasicInvocationHandler.invoke(BasicInvocationHandler.java:597) at net.jini.jeri.AtomicInvocationHandler.invoke(AtomicInvocationHandler.java:315) at com.sun.proxy.$Proxy11.getLocator(Unknown Source) at org.apache.river.reggie.proxy.RegistrarProxy.getLocator(RegistrarProxy.java:307) Caused by: java.io.IOException: request aborted by remote endpoint at org.apache.river.jeri.internal.mux.MuxInputStream.read(MuxInputStream.java:93) at net.jini.jeri.connection.ConnectionManager$Outbound$Input.read(ConnectionManager.java:591) at net.jini.jeri.BasicObjectEndpoint.executeCall(BasicObjectEndpoint.java:524) at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:888) ... 16 more [21:52:18.221] [(JSK) mux reader] [WARN ] net.jini.jeri.connection.mux - mux reader thread dying, unexpected exception java.lang.StackOverflowError: null at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140) at org.apache.river.jeri.internal.mux.StreamConnectionIO$1.read(StreamConnectionIO.java:372) at org.apache.river.jeri.internal.mux.StreamConnectionIO$Reader.run(StreamConnectionIO.java:277) at org.apache.river.thread.ThreadPool$Task.run(ThreadPool.java:172) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829)

I am using the below config to start reggie service and the below log is generated. So I assume lookup service is started correctly.

org.apache.river.reggie.discovery - Reggie Unicast Discovery listening on port 4,160 org.apache.river.reggie.service - started Reggie: f66fd086-7317-4f12-a0af-456f0abeda10, [com.example.group], ConstrainableLookupLocator[[jini://bharath:4160/], [null]]

reggie service configuration file

import net.jini.jeri.BasicILFactory; import net.jini.jeri.BasicJeriExporter; import net.jini.jeri.tcp.TcpServerEndpoint; import net.jini.jeri.AtomicILFactory; import org.apache.river.reggie.proxy.Registrar;

start {
discoveryGroup="com.example.group"; }

org.apache.river.reggie { initialLookupGroups = new String[] {start.discoveryGroup}; initialMemberGroups = new String[] {start.discoveryGroup}; }

It was working when I used it last year. Am I missing something?

Thanks Bharath

[pfirmstone]

Hi Bharath,

My Apologies for the late reply.

Which version of Java are you using?

I had to make changes recently regarding changes to ByteBuffer and Buffer in different Java versions, so it looks like I've missed something.  Currently I compile on Java 8 and run on later versions.

Thanks,

Peter.

On 23/10/2021 2:36 am, bharathkumara wrote:

I have built the latest version of JGDMS locally and am able to run the lookup service without any issues. But when we use LookupDiscovery to discover lookup services, ClassCastException is thrown. I have used the System property org.apache.river.jeri.tcp.useNIO=true to use NIO with TCPServerEndpoint.

The following exception is thrown while discovering registrars.

java.lang.ClassCastException: class [Ljava.nio.Buffer; cannot be cast to class [Ljava.nio.ByteBuffer; ([Ljava.nio.Buffer; and [Ljava.nio.ByteBuffer; are in module java.base of loader 'bootstrap') at org.apache.river.jeri.internal.mux.SocketChannelConnectionIO.asyncSend(SocketChannelConnectionIO.java:159) at org.apache.river.jeri.internal.mux.Mux.asyncSendData(Mux.java:619) at org.apache.river.jeri.internal.mux.MuxOutputStream.writeBuffer(MuxOutputStream.java:237) at org.apache.river.jeri.internal.mux.MuxOutputStream.close(MuxOutputStream.java:120) at java.base/java.io.ObjectOutputStream$BlockDataOutputStream.close(ObjectOutputStream.java:1834) at java.base/java.io.ObjectOutputStream.close(ObjectOutputStream.java:745) at org.apache.river.api.io.AtomicMarshalOutputStream.close(AtomicMarshalOutputStream.java:129) at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:861) at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethod(BasicInvocationHandler.java:728) at net.jini.jeri.BasicInvocationHandler.invoke(BasicInvocationHandler.java:597) at net.jini.jeri.AtomicInvocationHandler.invoke(AtomicInvocationHandler.java:315) at com.sun.proxy.$Proxy11.getLocator(Unknown Source) at org.apache.river.reggie.proxy.RegistrarProxy.getLocator(RegistrarProxy.java:

*Is this issue related to it?

117 https://github.com/pfirmstone/JGDMS/issues/117*

I have used JDK 11 to build and run it.

I tried to run it without useNIO option and UnmarshalException is thrown.

java.rmi.UnmarshalException: exception unmarshalling response; nested exception is: java.io.IOException: request aborted by remote endpoint at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:929) at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethod(BasicInvocationHandler.java:728) at net.jini.jeri.BasicInvocationHandler.invoke(BasicInvocationHandler.java:597) at net.jini.jeri.AtomicInvocationHandler.invoke(AtomicInvocationHandler.java:315) at com.sun.proxy.$Proxy11.getLocator(Unknown Source) at org.apache.river.reggie.proxy.RegistrarProxy.getLocator(RegistrarProxy.java:307) Caused by: java.io.IOException: request aborted by remote endpoint at org.apache.river.jeri.internal.mux.MuxInputStream.read(MuxInputStream.java:93) at net.jini.jeri.connection.ConnectionManager$Outbound$Input.read(ConnectionManager.java:591) at net.jini.jeri.BasicObjectEndpoint.executeCall(BasicObjectEndpoint.java:524) at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:888) ... 16 more [21:52:18.221] [(JSK) mux reader] [WARN ] net.jini.jeri.connection.mux

• mux reader thread dying, unexpected exception java.lang.StackOverflowError: null at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140) at org.apache.river.jeri.internal.mux.StreamConnectionIO$1.read(StreamConnectionIO.java:372) at org.apache.river.jeri.internal.mux.StreamConnectionIO$Reader.run(StreamConnectionIO.java:277) at org.apache.river.thread.ThreadPool$Task.run(ThreadPool.java:172) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829)

I am using the below config to start reggie service and the below log is generated. So I assume lookup service is started correctly.

org.apache.river.reggie.discovery - Reggie Unicast Discovery listening on port 4,160 org.apache.river.reggie.service - started Reggie: f66fd086-7317-4f12-a0af-456f0abeda10, [com.example.group], ConstrainableLookupLocator[[jini://bharath:4160/], [null]]

reggie service configuration file

import net.jini.jeri.BasicILFactory; import net.jini.jeri.BasicJeriExporter; import net.jini.jeri.tcp.TcpServerEndpoint; import net.jini.jeri.AtomicILFactory; import org.apache.river.reggie.proxy.Registrar;

start { discoveryGroup="com.example.group"; }

org.apache.river.reggie { initialLookupGroups = new String[] {start.discoveryGroup}; initialMemberGroups = new String[] {start.discoveryGroup}; }

It was working when I used it last year. Am I missing something?

Thanks Bharath

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pfirmstone/JGDMS/issues/141, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYWKMQWJF2R6BSZTOREKXDUIGHJTANCNFSM5GQ6H2UA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[bharathkumara]

Thanks Peter for the reply. I used Java 11 to build and run JGDMS.

Thanks, Bharath

[pfirmstone]

Hi  Bharath,

I've patched the issue if you want to give it another try.

Didn't pay close enough attention to array types in the original fix.

Cheers,

Peter.

On 28/11/2021 11:04 pm, bharathkumara wrote:

Thanks Peter for the reply. I used Java 11 to build and run JGDMS.

Thanks, Bharath

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pfirmstone/JGDMS/issues/141#issuecomment-981082154, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYWKMQK6UJ2P22Z4LJHL2LUOISERANCNFSM5GQ6H2UA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[pfirmstone]

StackOverflowError needs further investigation. I currently build using Java 8 and test on later versions. Have you got more information on how to reproduce the error?

[bharathkumara]

Thank you Peter. I have pulled the latest code and built it using Java 11. I encountered the same issue. When I removed the fix for https://github.com/pfirmstone/JGDMS/issues/117 (8th July 2020) It is working as expected. For Java 11, we may not require these changes for discovery to work. But Java 8 may require the fix.

Thanks, Bharath

[pfirmstone]

Thanks Bharath,

Can you send me a stack trace mate?   Just want to see where the ClassCastException is being thrown, I've reverted the Buffer[] array type changes, we're only using ByteBuffer[] array types now.  All Buffer instances are ByteBuffer object types, so we shouldn't be getting the CCE there.  Did you remember to mvn clean first?

Or are you still experiencing the Stack overflow error?

Cheers,

Peter.

On 13/12/2021 1:53 pm, bharathkumara wrote:

Thank you Peter. I have pulled the latest code and built it using Java

11. I encountered the same issue. When I removed the fix for #117 https://github.com/pfirmstone/JGDMS/issues/117 (8th July 2020) It is working as expected. For Java 11, we may not require these changes for discovery to work. But Java 8 may require the fix.

Thanks, Bharath

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/pfirmstone/JGDMS/issues/141#issuecomment-992085038, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYWKMSK6FGDVOLIALOBCSLUQVU4PANCNFSM5GQ6H2UA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[bharathkumara]

jgdms_discovery_stacktrace.txt

Thanks Peter, I attached the stack trace file. Here I have collected it without org.apache.river.jeri.tcp.useNIO=true option.

[bharathkumara]

Peter, I have attached the stack trace with org.apache.river.jeri.tcp.useNIO=true option.

jgdms_discovery_stacktrace_nio.txt

Thanks, Bharath

[bharathkumara]

Hi Peter I have re-tested this issue with the latest code using AdoptOpenJDK 11. I found that the latest code is working without any errors. I am using Ubuntu Linux 21.04.

openjdk version "11.0.11" 2021-04-20 OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9) Eclipse OpenJ9 VM AdoptOpenJDK-11.0.11+9 (build openj9-0.26.0, JRE 11 Linux amd64-64-Bit Compressed References 20210421_975 (JIT enabled, AOT enabled) OpenJ9 - b4cc246d9 OMR - 162e6f729 JCL - 7796c80419 based on jdk-11.0.11+9)

When I use Temurin JDK 11, I could see all these errors and discovery is not working.

openjdk version "11.0.13" 2021-10-19 OpenJDK Runtime Environment Temurin-11.0.13+8 (build 11.0.13+8) OpenJDK 64-Bit Server VM Temurin-11.0.13+8 (build 11.0.13+8, mixed mode)

I suspect the problem is with that JDK.

[bharathkumara]

Thank you Peter for your kind support. Discovery is working with AdoptOpenJDK 11 without any issues. When we use Temurin JDK, there are exceptions and discovery is not working. It may be an issue with the particular JDK.

jgdms_discovery_success.txt

[pfirmstone]

Thanks Bharath,

I reviewed the changes and couldn't find anything that caused the OME. Thanks for reporting back and identifying the ClassCastException. I may yet replace some of the uses of Buffer with ByteBuffer in the API, as it expects ByteBuffer, but uses the method signatures of Buffer, due to changes made in Java 9. Another option might be to create a wrapper class that extends Buffer and encapsulates ByteBuffer, so we can avoid multiple casts, as this makes the code brittle.

[pfirmstone]

Hi Bharath,

I noticed you're using Atomic serialization and OSGi, interested in any feedback or improvements you'd like to see.

Cheers,

Peter.

On 14/12/2021 3:01 am, bharathkumara wrote:

Thank you Peter for your kind support. Discovery is working with AdoptOpenJDK 11 without any issues. When we use Temurin JDK, there are exceptions and discovery is not working. It may be an issue with the particular version.

jgdms_discovery_success.txt https://github.com/pfirmstone/JGDMS/files/7705561/jgdms_discovery_success.txt

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/pfirmstone/JGDMS/issues/141#issuecomment-992680124, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYWKMU4Q7YL2YPGNBJM3P3UQYRFVANCNFSM5GQ6H2UA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.