java.sql.SQLRecoverableException: ORA-17002: I/O error: , Authentication lapse 0 ms.

[kartikshingala78]

Before reporting an issue

• [X] I have read and understood the above terms for submitting issues, and I understand that my issue may be closed without action if I do not follow them.

Area

dist/quarkus

Describe the bug

During upgrade from keycloak 22.0.5 to keycloak 25.0.6. I am getting errors :-

1) java.sql.SQLRecoverableException: ORA-17002: I/O error: , Authentication lapse 0 ms

2) 2024-10-11 17:23:06,114 ERROR [org.jgroups.protocols.UDP] (mcast-receiver-6,SvistaLStest-15943) JGRP000030: SvistaLStest-15943: failed handling incoming message: java.lang.IllegalArgumentException: invalid magic number -31438; needs to be in range [0..100]

openjdk version "21.0.4" 2024-07-16 ojdbc11-23.5.0.24.07.jar

If I rollback to previous version everything works.

I have attached output of show-config and keycloak logs.

keycloak.log.txt kc.sh show-config.txt

Version

25.0.6

Regression

• [X] The issue is a regression

Expected behavior

Keycloak should be able to connect to Oracle RAC DB without errors.

Actual behavior

Keycloak is not able to connect to Oracle RAC DB.

How to Reproduce?

upgrade from :- openjdk version "17.0.2" 2022-01-18 keycloak 22.0.5 Upgrade to :- openjdk version "21.0.4" 2024-07-16 ojdbc11-23.5.0.24.07.jar keycloak 25.0.6

Anything else?

No response

[shawkins]

java.sql.SQLRecoverableException: ORA-17002: I/O error: , Authentication lapse 0 ms

Ideally this is not related to the Keycloak upgrade. A quick search seems to suggest that there's something wrong in your environment. cc @yrodiere

2024-10-11 17:23:06,114 ERROR [org.jgroups.protocols.UDP] (mcast-receiver-6,SvistaLStest-15943) JGRP000030:

This looks like you are attempting to perform a rolling upgrade and are receiving messages from the older keycloaks. This is not supported. cc @ryanemerson

Please confirm how you are performing this upgrade.

[kartikshingala78]

@shawkins ,

Ideally this is not related to the Keycloak upgrade. A quick search seems to suggest that there's something wrong in your environment. cc @yrodiere

You are right this was Oracle SCAN listener issue and it is fixed now. Not getting this error anymore.

Please confirm how you are performing this upgrade.

Steps followed are as follows :-

1) Export realm from keycloak 22. kc.sh export --file ./svip_realm_keycloak22.json --realm SmartVista

2) Backup keycloak 22 directory.

3) Extract keycloak 25 binaries.

4) Copy keycloak.conf from keycloak 22 to keycloak 25.

5) Import realm to keycloak exported kc.sh import --file ./svip_realm_keycloak22.json

6) Start keycloak kc.sh start

It was giving errors :- java.sql.SQLRecoverableException: ORA-17002: I/O error: , Authentication lapse 0 ms

I tried to update it manually using below command :- kc.sh start --spi-connections-jpa-default-migration-strategy=update

After fixing listener issue "ORA-17002: I/O error:" is gone

But below error is still there :-

ERROR [org.jgroups.protocols.UDP] (mcast-receiver-6,SvistaLStest-15943) JGRP000030: SvistaLStest-15943: failed handling incoming message: java.lang.IllegalArgumentException: invalid magic number -31438; needs to be in range [0..100]

Seems issue is caused by below command but I am not sure how to fix it.

kc.sh start --spi-connections-jpa-default-migration-strategy=update

[shawkins]

Are you stopping the keycloak 22 instance before starting the new one? I wouldn't expect there to be an invalid message coming over UDP unless both versions were joining the cluster at the same time.

[kartikshingala78]

Yes, keycloak 22 was stopped before starting keycloak 25. And this is is standalone keycloak it is not a cluster.

We are still getting these messages in logs and only keycloak 25 is running at this time.

[shawkins]

Added core-shared, I'm not what else would cause a jgroups error like that.

[ryanemerson]

Are you stopping the keycloak 22 instance before starting the new one? I wouldn't expect there to be an invalid message coming over UDP unless both versions were joining the cluster at the same time.

This is what I would expect as well

[shawkins]

Based upon the logs, it does not appear there is another member in the cluster:

2024-10-11 16:24:44,277 INFO  [org.jgroups.protocols.pbcast.GMS] (Thread-5) SvistaLStest-5469: no members discovered after 2003 ms: creating cluster as coordinator
2024-10-11 16:24:44,285 INFO  [org.infinispan.CLUSTER] (Thread-5) ISPN000094: Received new cluster view for channel ISPN: [SvistaLStest-5469|0] (1) [SvistaLStest-5469]
2024-10-11 16:24:44,344 INFO  [org.infinispan.CLUSTER] (Thread-5) ISPN000079: Channel `ISPN` local address is `SvistaLStest-5469`, physical addresses are `[172.18.104.116:50166]

Copy keycloak.conf from keycloak 22 to keycloak 25.

@kartikshingala78 Can you provide the show-config output? And to double check is there anything that would alter the dependencies in keycloak, such as having custom providers jars?

[keycloak-github-bot[bot]]

Thanks for reporting this issue, but there is insufficient information or lack of steps to reproduce.

Please provide additional details, otherwise this issue will be automatically closed within 14 days.

[kartikshingala78]

@shawkins, Yes, it is stand alone instance there is no other member.

I have attached output of show-config to :- https://github.com/user-attachments/files/17343731/kc.sh.show-config.txt

There are no custom jar files in providers.

[shawkins]

@kartikshingala78 thank you for the additonal information.

@ryanemerson what else would be good to have here?

[ryanemerson]

@ryanemerson what else would be good to have here?

Tbh I don't know at this point. As you point out there's no other members joining the cluster, but it still appears that packets of an unknown format are being received over UDP. Could it be from a non-keycloak process? @pruivo any thoughts?

[pruivo]

It could be an external process with some lucky packet or another JGroups process running on the network.

2024-10-11 16:24:45,895 ERROR [org.jgroups.protocols.UDP] (mcast-receiver-6,SvistaLStest-5469) JGRP000030: SvistaLStest-5469: failed handling incoming message: java.lang.IllegalArgumentException: invalid magic number -31438; needs to be in range [0..100]
        at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:136)
        at org.jgroups.util.Util.readSizeStreamable(Util.java:2004)
        at org.jgroups.ObjectMessage.readPayload(ObjectMessage.java:144)
        at org.jgroups.BaseMessage.readFrom(BaseMessage.java:317)
        at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1347)
        at org.jgroups.protocols.TP.receive(TP.java:1303)
        at org.jgroups.protocols.UDP$PacketReceiver.run(UDP.java:809)
        at java.base/java.lang.Thread.run(Thread.java:1583)

This bothers me because:

• JGroups version decoded from the packet and matches the current version.
• Flags, Source ID, Destination ID, and Message headers all decoded without issues (!!)

A network packet tool may help to figure it out where it comes from. @belaban may have more insights.

[belaban]

This is probably an incompatibility between JGroups versions used by KC 22 and 25. It was mentioned that this is an unclustered deployment, but I guess that's not correct, or else we'd not be seeing these warning messages...?

[ahus1]

If this is a standalone instance, then disable the clustering. Otherwise it might interfere with other standalone instances that might be discovered accidentally using multicast.

To disable it, start Keycloak with --cache=local as described in https://www.keycloak.org/server/caching

Then this issue should go away. In Keycloak 26.1 we plan to have a better way of discovering other nodes which should give errors with UDP or multicast (see #34102 which implements JDBC_PING).

[ahus1]

Please comment here if this fixes your issue, and close the issue if it does. Thanks!