search

bitsofinfo / hazelcast-docker-swarm-discovery-spi

Docker Swarm based discovery strategy SPI for Hazelcast enabled applications
Apache License 2.0
39 stars 33 forks source link

Problem setting-up embedded hazelcast-jet cluster in docker swarm(overlay network) #51

Open mahendya1002 opened 4 years ago

mahendya1002 commented 4 years ago

Hi,

I am trying to form a hazelcast jet cluster in my docker environment in embedded way. And for the same i tried following Option 1: Local network binding via SwarmMemberAddressProvider

Following are the dependency i am referring too.

        <dependency>
            <groupId>com.hazelcast.jet</groupId>
            <artifactId>hazelcast-jet</artifactId>
            <version>3.1</version>
        </dependency>
        <dependency>
            <groupId>com.hazelcast.jet</groupId>
            <artifactId>hazelcast-jet-kafka</artifactId>
            <version>3.1</version>
        </dependency>
        <dependency>
            <groupId>org.bitsofinfo</groupId>
            <artifactId>hazelcast-docker-swarm-discovery-spi</artifactId>
            <version>1.0-RC8</version> <!--  Use 1.0.RC3 for Hazelcast < 3.8.x -->
            <exclusions>
                <exclusion>
                    <groupId>com.hazelcast</groupId>
                    <artifactId>hazelcast</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

Containers are coming up but i don't see Cluster formation and any error logs. Any help in this regard is appreciated.

Here is my hazelcast-jet-member-default.xml file content

<?xml version="1.0" encoding="UTF-8"?>
<!--
  ~ Copyright (c) 2008-2019, Hazelcast, Inc. All Rights Reserved.
  ~
  ~ Licensed under the Apache License, Version 2.0 (the "License");
  ~ you may not use this file except in compliance with the License.
  ~ You may obtain a copy of the License at
  ~
  ~ http://www.apache.org/licenses/LICENSE-2.0
  ~
  ~ Unless required by applicable law or agreed to in writing, software
  ~ distributed under the License is distributed on an "AS IS" BASIS,
  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  ~ See the License for the specific language governing permissions and
  ~ limitations under the License.
  -->

<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-3.12.xsd"
           xmlns="http://www.hazelcast.com/schema/config"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <group>
        <name>jet</name>
    </group>
    <properties>
        <property name="hazelcast.discovery.enabled">true</property>
        <property name="hazelcast.shutdownhook.enabled">true</property>
        <property name="hazelcast.socket.bind.any">false</property>
    </properties>
    <license-key>***</license-key>
    <network>
        <port auto-increment="true" port-count="100">5701</port>
        <interfaces enabled="false">
            <interface>${hazelcastInterface}</interface>
        </interfaces>
        <ssl enabled="false"/>
        <!-- Enable a Docker Swarm based MemberAddressProvider (Hazelcast 3.9+ only) -->
        <member-address-provider enabled="true">
            <class-name>org.bitsofinfo.hazelcast.discovery.docker.swarm.SwarmMemberAddressProvider</class-name>
        </member-address-provider>

        <join>
            <multicast enabled="false"/>
            <aws enabled="false"/>
            <tcp-ip enabled="false"/>

            <!-- Enable a Docker Swarm based discovery strategy -->
            <discovery-strategies>
                <discovery-strategy enabled="true"
                                    class="org.bitsofinfo.hazelcast.discovery.docker.swarm.DockerSwarmDiscoveryStrategy">
                    <properties>
                        <property name="docker-network-names">test_env_network</property>
                        <property name="docker-service-names">test</property>
                        <property name="docker-service-labels">test_env</property>
                        <property name="swarm-mgr-uri">https://...:2375</property>
                        <property name="skip-verify-ssl">true</property>
                        <property name="hazelcast-peer-port">5701</property>
                    </properties>

                </discovery-strategy>
            </discovery-strategies>
        </join>
    </network>
</hazelcast>

Container1 logs are

Mar 02, 2020 7:49:20 AM com.hazelcast.jet.config.JetConfig
INFO: jet.home is /release/rls/resources/current
Mar 02, 2020 7:49:20 AM com.hazelcast.jet.config.JetConfig
INFO: Configuring Hazelcast Jet from '/release/rls/resources/current/hazelcast-jet-member-default.xml'.
Mar 02, 2020 7:49:20 AM com.hazelcast.config.ConfigReplacerHelper
WARNING: Could not find a replacement for '${hazelcastInterface}' on node 'null'
Mar 02, 2020 7:49:21 AM com.hazelcast.config.AbstractConfigLocator
INFO: Loading 'hazelcast-jet-member-default.xml' from the classpath.
Mar 02, 2020 7:49:21 AM com.hazelcast.instance.AddressPicker
INFO: [LOCAL] [jet] [3.1] Prefer IPv4 stack is true, prefer IPv6 addresses is false
Mar 02, 2020 7:49:21 AM com.hazelcast.instance.AddressPicker
INFO: [LOCAL] [jet] [3.1] Picked [172.18.0.3]:5701, using socket ServerSocket[addr=/172.18.0.3,localport=5701], bind any local is false
Mar 02, 2020 7:49:21 AM com.hazelcast.system
INFO: [172.18.0.3]:5701 [jet] [3.1] Hazelcast Jet 3.1 (20190624 - 000ced7) starting at [172.18.0.3]:5701
Mar 02, 2020 7:49:21 AM com.hazelcast.system
INFO: [172.18.0.3]:5701 [jet] [3.1]
    o   o   o   o---o o---o o     o---o   o   o---o o-o-o        o o---o o-o-o
    |   |  / \     /  |     |     |      / \  |       |          | |       |
    o---o o---o   o   o-o   |     o     o---o o---o   |          | o-o     |
    |   | |   |  /    |     |     |     |   |     |   |      \   | |       |
    o   o o   o o---o o---o o---o o---o o   o o---o   o       o--o o---o   o
Mar 02, 2020 7:49:21 AM com.hazelcast.system
INFO: [172.18.0.3]:5701 [jet] [3.1] Copyright (c) 2008-2019, Hazelcast, Inc. All Rights Reserved.
Mar 02, 2020 7:49:21 AM com.hazelcast.spi.impl.operationservice.impl.BackpressureRegulator
INFO: [172.18.0.3]:5701 [jet] [3.1] Backpressure is disabled
Mar 02, 2020 7:49:22 AM com.hazelcast.instance.Node
**INFO: [172.18.0.3]:5701 [jet] [3.1] Activating Discovery SPI Joiner**
Mar 02, 2020 7:49:22 AM com.hazelcast.jet.impl.metrics.JetMetricsService
INFO: [172.18.0.3]:5701 [jet] [3.1] Configuring metrics collection, collection interval=5 seconds, retention=5 seconds, publishers=[Management Center Publisher, JMX Publisher]
Mar 02, 2020 7:49:22 AM com.hazelcast.jet.impl.JetService
INFO: [172.18.0.3]:5701 [jet] [3.1] Setting number of cooperative threads and default parallelism to 4
Mar 02, 2020 7:49:22 AM com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl
INFO: [172.18.0.3]:5701 [jet] [3.1] Starting 4 partition threads and 3 generic threads (1 dedicated for priority tasks)
Mar 02, 2020 7:49:22 AM com.hazelcast.internal.diagnostics.Diagnostics
INFO: [172.18.0.3]:5701 [jet] [3.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
Mar 02, 2020 7:49:22 AM com.hazelcast.core.LifecycleService
INFO: [172.18.0.3]:5701 [jet] [3.1] [172.18.0.3]:5701 is STARTING
Mar 02, 2020 7:49:27 AM com.hazelcast.internal.cluster.ClusterService
INFO: [172.18.0.3]:5701 [jet] [3.1]

**Members {size:1, ver:1} [
    Member [172.18.0.3]:5701 - bd2c5e10-5f81-4396-9913-36e81e201fe9 this
]**

Mar 02, 2020 7:49:27 AM com.hazelcast.core.LifecycleService
INFO: [172.18.0.3]:5701 [jet] [3.1] [172.18.0.3]:5701 is STARTED

Container2 logs are

Mar 02, 2020 7:49:18 AM com.hazelcast.jet.config.JetConfig
INFO: jet.home is /release/rls/resources/current
Mar 02, 2020 7:49:18 AM com.hazelcast.jet.config.JetConfig
INFO: Configuring Hazelcast Jet from '/release/rls/resources/current/hazelcast-jet-member-default.xml'.
Mar 02, 2020 7:49:18 AM com.hazelcast.config.ConfigReplacerHelper
WARNING: Could not find a replacement for '${hazelcastInterface}' on node 'null'
Mar 02, 2020 7:49:18 AM com.hazelcast.config.AbstractConfigLocator
INFO: Loading 'hazelcast-jet-member-default.xml' from the classpath.
Mar 02, 2020 7:49:18 AM com.hazelcast.instance.AddressPicker
INFO: [LOCAL] [jet] [3.1] Prefer IPv4 stack is true, prefer IPv6 addresses is false
Mar 02, 2020 7:49:18 AM com.hazelcast.instance.AddressPicker
INFO: [LOCAL] [jet] [3.1] Picked [172.18.0.5]:5701, using socket ServerSocket[addr=/172.18.0.5,localport=5701], bind any local is false
Mar 02, 2020 7:49:18 AM com.hazelcast.system
INFO: [172.18.0.5]:5701 [jet] [3.1] Hazelcast Jet 3.1 (20190624 - 000ced7) starting at [172.18.0.5]:5701
Mar 02, 2020 7:49:18 AM com.hazelcast.system
INFO: [172.18.0.5]:5701 [jet] [3.1]
    o   o   o   o---o o---o o     o---o   o   o---o o-o-o        o o---o o-o-o
    |   |  / \     /  |     |     |      / \  |       |          | |       |
    o---o o---o   o   o-o   |     o     o---o o---o   |          | o-o     |
    |   | |   |  /    |     |     |     |   |     |   |      \   | |       |
    o   o o   o o---o o---o o---o o---o o   o o---o   o       o--o o---o   o
Mar 02, 2020 7:49:18 AM com.hazelcast.system
INFO: [172.18.0.5]:5701 [jet] [3.1] Copyright (c) 2008-2019, Hazelcast, Inc. All Rights Reserved.
Mar 02, 2020 7:49:19 AM com.hazelcast.spi.impl.operationservice.impl.BackpressureRegulator
INFO: [172.18.0.5]:5701 [jet] [3.1] Backpressure is disabled
Mar 02, 2020 7:49:19 AM com.hazelcast.instance.Node
**INFO: [172.18.0.5]:5701 [jet] [3.1] Activating Discovery SPI Joiner**
Mar 02, 2020 7:49:19 AM com.hazelcast.jet.impl.metrics.JetMetricsService
INFO: [172.18.0.5]:5701 [jet] [3.1] Configuring metrics collection, collection interval=5 seconds, retention=5 seconds, publishers=[Management Center Publisher, JMX Publisher]
Mar 02, 2020 7:49:19 AM com.hazelcast.jet.impl.JetService
INFO: [172.18.0.5]:5701 [jet] [3.1] Setting number of cooperative threads and default parallelism to 4
Mar 02, 2020 7:49:19 AM com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl
INFO: [172.18.0.5]:5701 [jet] [3.1] Starting 4 partition threads and 3 generic threads (1 dedicated for priority tasks)
Mar 02, 2020 7:49:19 AM com.hazelcast.internal.diagnostics.Diagnostics
INFO: [172.18.0.5]:5701 [jet] [3.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
Mar 02, 2020 7:49:19 AM com.hazelcast.core.LifecycleService
INFO: [172.18.0.5]:5701 [jet] [3.1] [172.18.0.5]:5701 is STARTING
Mar 02, 2020 7:49:24 AM com.hazelcast.internal.cluster.ClusterService
INFO: [172.18.0.5]:5701 [jet] [3.1]

**Members {size:1, ver:1} [
    Member [172.18.0.5]:5701 - 5e72812f-6ce4-4a9f-aa4e-ab032aea881d this
]**

Mar 02, 2020 7:49:24 AM com.hazelcast.core.LifecycleService
INFO: [172.18.0.5]:5701 [jet] [3.1] [172.18.0.5]:5701 is STARTED
cangencer commented 4 years ago

@mahendya1002 are you a Jet enterprise user?