search

dapr / components-contrib

Community driven, reusable components for distributed apps
Apache License 2.0
533 stars 468 forks source link

Add support for Hazelcast 4.X, 5.X in state store #3072

Open prtk-fyndna opened 11 months ago

prtk-fyndna commented 11 months ago

Expected Behavior

Dapr application starts with Hazelcast as a state store.

Actual Behavior

State store component loading failed, resulting in application termination

Steps to Reproduce the Problem

Dapr CLI version: 1.3.0 Dapr Runtime version: 1.11.2 Hazelcast version: 5.3.1

  1. Create component.yml for hazelcast state store
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: transaction-state-store
spec:
  type: state.hazelcast
  version: v1
  metadata:
    - name: hazelcastServers
      value: localhost:5701
    - name: hazelcastMap
      value: transactions
  1. Start Hazelcast Server
docker run -e JAVA_OPTS="-Dhazelcast.local.publicAddress=127.0.0.1:5701" -p 5701:5701 hazelcast/hazelcast
  1. Start dapr application with hazelcast component

Logs

Dapr

INFO[0000] Loading components…                           app_id=hazelcast-demo-producer instance=Prateeks-MacBook-Air.local scope=dapr.runtime type=log ver=1.11.2
INFO[0000] Waiting for all outstanding components to be processed  app_id=hazelcast-demo-producer instance=Prateeks-MacBook-Air.local scope=dapr.runtime type=log ver=1.11.2
2023/08/09 16:41:16 github.com/hazelcast/hazelcast-go-client/internal.(*lifecycleService).fireLifecycleEvent
INFO: hz.client_1 [dev] [0.5-SNAPSHOT] HazelcastClient 0.5-SNAPSHOT is STARTING
2023/08/09 16:41:16 github.com/hazelcast/hazelcast-go-client/internal.(*clusterService).connectToAddress
INFO: hz.client_1 [dev] [0.5-SNAPSHOT] Trying to connect to localhost:5701 as owner member.
2023/08/09 16:41:16 github.com/hazelcast/hazelcast-go-client/internal.(*Connection).close
WARN: hz.client_1 [dev] [0.5-SNAPSHOT] Connection : %!v(PANIC=String method: interface conversion: interface is nil, not core.Address)  closed, err:  EOF
2023/08/09 16:41:17 github.com/hazelcast/hazelcast-go-client/internal.(*clusterService).connectToAddress
WARN: hz.client_1 [dev] [0.5-SNAPSHOT] Error during initial connection to localhost:5701 error: packet is not sent

2023/08/09 16:41:20 github.com/hazelcast/hazelcast-go-client/internal.(*clusterService).connectToAddress
INFO: hz.client_1 [dev] [0.5-SNAPSHOT] Trying to connect to localhost:5701 as owner member.
2023/08/09 16:41:21 github.com/hazelcast/hazelcast-go-client/internal.(*Connection).close
WARN: hz.client_1 [dev] [0.5-SNAPSHOT] Connection : %!v(PANIC=String method: interface conversion: interface is nil, not core.Address)  closed, err:  EOF
WARN[0005] Error processing component, daprd process will exit gracefully  app_id=hazelcast-demo-producer instance=Prateeks-MacBook-Air.local scope=dapr.runtime type=log ver=1.11.2
FATA[0005] process component transaction-state-store error: [INIT_COMPONENT_FAILURE]: initialization error occurred for transaction-state-store (state.hazelcast/v1): init timeout for component transaction-state-store exceeded after 5s  app_id=hazelcast-demo-producer instance=Prateeks-MacBook-Air.local scope=dapr.runtime type=log ver=1.11.2
❌  The daprd process exited with error code: exit status 1

Hazelcast

2023-08-09 11:11:16,947 [ WARN] [hz.mystifying_cartwright.IO.thread-in-0] [c.h.i.s.t.TcpServerConnection]: [127.0.0.1]:5701 [dev] [5.3.1] Connection[id=1, /172.17.0.5:5701->/172.17.0.1:61926, qualifier=null, endpoint=null, remoteUuid=null, alive=false, connectionType=NONE, planeIndex=-1] closed. Reason: Exception in Connection[id=1, /172.17.0.5:5701->/172.17.0.1:61926, qualifier=null, endpoint=null, remoteUuid=null, alive=true, connectionType=NONE, planeIndex=-1], thread=hz.mystifying_cartwright.IO.thread-in-0
java.lang.IllegalStateException: Unknown protocol: CB2
at com.hazelcast.internal.server.tcp.UnifiedProtocolDecoder.onRead(UnifiedProtocolDecoder.java:132) ~[hazelcast-5.3.1.jar:5.3.1]
at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:137) ~[hazelcast-5.3.1.jar:5.3.1]
at com.hazelcast.internal.networking.nio.NioPipeline.lambda$start$0(NioPipeline.java:127) ~[hazelcast-5.3.1.jar:5.3.1]
at com.hazelcast.internal.networking.nio.NioThread.processTaskQueue(NioThread.java:355) [hazelcast-5.3.1.jar:5.3.1]
at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:290) [hazelcast-5.3.1.jar:5.3.1]
at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249) [hazelcast-5.3.1.jar:5.3.1]
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:111) [hazelcast-5.3.1.jar:5.3.1]
2023-08-09 11:11:21,011 [ WARN] [hz.mystifying_cartwright.IO.thread-in-1] [c.h.i.s.t.TcpServerConnection]: [127.0.0.1]:5701 [dev] [5.3.1] Connection[id=2, /172.17.0.5:5701->/172.17.0.1:61930, qualifier=null, endpoint=null, remoteUuid=null, alive=false, connectionType=NONE, planeIndex=-1] closed. Reason: Exception in Connection[id=2, /172.17.0.5:5701->/172.17.0.1:61930, qualifier=null, endpoint=null, remoteUuid=null, alive=true, connectionType=NONE, planeIndex=-1], thread=hz.mystifying_cartwright.IO.thread-in-1
java.lang.IllegalStateException: Unknown protocol: CB2
at com.hazelcast.internal.server.tcp.UnifiedProtocolDecoder.onRead(UnifiedProtocolDecoder.java:132) ~[hazelcast-5.3.1.jar:5.3.1]
at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:137) ~[hazelcast-5.3.1.jar:5.3.1]
at com.hazelcast.internal.networking.nio.NioPipeline.lambda$start$0(NioPipeline.java:127) ~[hazelcast-5.3.1.jar:5.3.1]
at com.hazelcast.internal.networking.nio.NioThread.processTaskQueue(NioThread.java:355) [hazelcast-5.3.1.jar:5.3.1]
at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:290) [hazelcast-5.3.1.jar:5.3.1]
at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249) [hazelcast-5.3.1.jar:5.3.1]
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:111) [hazelcast-5.3.1.jar:5.3.1]
shubham1172 commented 11 months ago

This seems to be due to Dapr using a really old version of the Hazelcast SDK.

A workaround is to use Hazelcast 3.12.12 -

docker run -e JAVA_OPTS="-Dhazelcast.local.publicAddress=127.0.0.1:5701" -p 5701:5701 hazelcast/hazelcast:3.12.12

The underlying SDK needs to be upgraded to use 4.x/5.x Hazelcast versions. Note, this might break the existing users using 3.x clients, so we can consider an alpha v2 release with the latest SDK version.

berndverst commented 11 months ago

@shubham1172 Hazelcast is an Alpha component, and it is also hardly used (and we use a very old pre-release version of the SDK). As such I am ok with a breaking change in this component. At this point however, this will miss the 1.12 release and can go into 1.13. I do not support creating a separate version - this would bring in another dependency with memory and goroutine implications. So the existing component should just be updated.

If two versions are required I request that this is done via pluggable components. We will not include two component versions in the runtime itself.

As @shubham1172 said, at this time please use Hazelcast 3.X. In Dapr 1.13 we will look into upgrading this.

berndverst commented 10 months ago

We have cut the 1.12 release branch - so at this time we can start accepting a PR for the Hazelcast library upgrade against our components-contrib master branch.