Is it possible to deploy a docker stack without traefik?

[0lucasmoura]

I'm trying to deploy a service stack in a swarm cluster, and for some reason unknown reason the datanode can`t connect to the namenode. Since these services are in the same network, they are visible to each other, the problem lies somewhere in how hdfs connects with its nodes.

Following your examples I see that you use traefik to expose the services.

Is this swarm setup not possible without it? How does traefik work in this context?

---

log:

Windows PowerShell
Copyright (C) Microsoft Corporation. Todos os direitos reservados.

Experimente a nova plataforma cruzada PowerShell https://aka.ms/pscore6

PS D:\projects\swarm-spark> docker logs -f 82e49d34fbe505eab0aa52e6b08de2f05f16fe0edb70b0d3894e15336b5f28c2
Configuring core
 - Setting fs.defaultFS=hdfs://primary-namenode:8020
Configuring hdfs
 - Setting dfs.datanode.data.dir=file:///hadoop/dfs/data
 - Setting dfs.webhdfs.enabled=true
Configuring yarn
Configuring httpfs
Configuring kms
Configuring mapred
Configuring for multihomed network
2020-05-22 19:50:34,083 INFO datanode.DataNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG:   host = datanode/10.0.9.12
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 3.2.1
STARTUP_MSG:   classpath = /etc/hadoop:/opt/hadoop-3.2.1/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/paranamer-2.3.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/metrics-core-3.2.4.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jersey-core-1.19.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/stax2-api-3.1.4.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/netty-3.10.5.Final.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/commons-io-2.5.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jackson-core-2.9.8.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/zookeeper-3.4.13.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/guava-27.0-jre.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/commons-collections-3.2.2.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/token-provider-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/commons-math3-3.1.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/failureaccess-1.0.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jsch-0.1.54.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/commons-lang3-3.7.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/log4j-1.2.17.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/httpcore-4.4.10.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jackson-annotations-2.9.8.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerb-core-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/commons-logging-1.1.3.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/asm-5.0.4.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/checker-qual-2.5.2.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/commons-net-3.6.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jersey-servlet-1.19.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/hadoop-annotations-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/commons-cli-1.2.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerb-client-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/avro-1.7.7.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jersey-server-1.19.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/json-smart-2.3.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerby-util-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerb-server-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jackson-databind-2.9.8.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerb-common-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/accessors-smart-1.2.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jsr305-3.0.0.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/curator-framework-2.13.0.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jersey-json-1.19.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/commons-compress-1.18.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jsp-api-2.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/commons-codec-1.11.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerb-util-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jettison-1.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/gson-2.2.4.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/curator-client-2.13.0.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/dnsjava-2.1.7.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerby-config-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/httpclient-4.5.6.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/hadoop-auth-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/commons-text-1.4.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/re2j-1.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/error_prone_annotations-2.2.0.jar:/opt/hadoop-3.2.1/share/hadoop/common/lib/snappy-java-1.0.5.jar:/opt/hadoop-3.2.1/share/hadoop/common/hadoop-nfs-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/hadoop-common-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/common/hadoop-common-3.2.1-tests.jar:/opt/hadoop-3.2.1/share/hadoop/common/hadoop-kms-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/paranamer-2.3.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-io-2.5.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jackson-core-2.9.8.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-lang3-3.7.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/httpcore-4.4.10.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jackson-annotations-2.9.8.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/asm-5.0.4.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-net-3.6.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/hadoop-annotations-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/avro-1.7.7.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/json-smart-2.3.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jackson-databind-2.9.8.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/okio-1.6.0.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jettison-1.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/gson-2.2.4.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/dnsjava-2.1.7.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/httpclient-4.5.6.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/hadoop-auth-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/commons-text-1.4.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/re2j-1.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/error_prone_annotations-2.2.0.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/hadoop-hdfs-rbf-3.2.1-tests.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/hadoop-hdfs-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/hadoop-hdfs-native-client-3.2.1-tests.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/hadoop-hdfs-nfs-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/hadoop-hdfs-client-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/hadoop-hdfs-3.2.1-tests.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/hadoop-hdfs-native-client-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/hadoop-hdfs-rbf-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/hdfs/hadoop-hdfs-client-3.2.1-tests.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/lib/junit-4.11.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.2.1-tests.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/java-util-1.9.0.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/bcpkix-jdk15on-1.60.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/bcprov-jdk15on-1.60.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/fst-2.50.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.9.8.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/jersey-client-1.19.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/jackson-jaxrs-base-2.9.8.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.9.8.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/objenesis-1.0.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/guice-4.0.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/json-io-2.5.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-server-tests-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-server-router-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-client-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-common-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-submarine-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-server-common-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-services-api-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-registry-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-services-core-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.2.1.jar:/opt/hadoop-3.2.1/share/hadoop/yarn/hadoop-yarn-api-3.2.1.jar
STARTUP_MSG:   build = https://gitbox.apache.org/repos/asf/hadoop.git -r b3cbbb467e22ea829b3808f4b7b01d07e0bf3842; compiled by 'rohithsharmaks' on 2019-09-10T15:56Z
STARTUP_MSG:   java = 1.8.0_232
************************************************************/
2020-05-22 19:50:34,108 INFO datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT]
2020-05-22 19:50:34,984 INFO checker.ThrottledAsyncChecker: Scheduling a check for [DISK]file:/hadoop/dfs/data
2020-05-22 19:50:35,200 INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
2020-05-22 19:50:35,337 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2020-05-22 19:50:35,337 INFO impl.MetricsSystemImpl: DataNode metrics system started
2020-05-22 19:50:35,751 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2020-05-22 19:50:35,753 INFO datanode.BlockScanner: Initialized block scanner with targetBytesPerSec 1048576
2020-05-22 19:50:35,759 INFO datanode.DataNode: Configured hostname is datanode
2020-05-22 19:50:35,759 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2020-05-22 19:50:35,762 INFO datanode.DataNode: Starting DataNode with maxLockedMemory = 0
2020-05-22 19:50:35,820 INFO datanode.DataNode: Opened streaming server at /0.0.0.0:9866
2020-05-22 19:50:35,827 INFO datanode.DataNode: Balancing bandwidth is 10485760 bytes/s
2020-05-22 19:50:35,827 INFO datanode.DataNode: Number threads for balancing is 50
2020-05-22 19:50:35,898 INFO util.log: Logging initialized @3032ms
2020-05-22 19:50:36,143 INFO server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2020-05-22 19:50:36,158 INFO http.HttpRequestLog: Http request log for http.requests.datanode is not defined
2020-05-22 19:50:36,173 INFO http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2020-05-22 19:50:36,177 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
2020-05-22 19:50:36,178 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2020-05-22 19:50:36,178 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
2020-05-22 19:50:36,235 INFO http.HttpServer2: Jetty bound to port 40135
2020-05-22 19:50:36,237 INFO server.Server: jetty-9.3.24.v20180605, build timestamp: 2018-06-05T17:11:56Z, git hash: 84205aa28f11a4f31f2a3b86d1bba2cc8ab69827
2020-05-22 19:50:36,306 INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@2ad48653{/logs,file:///opt/hadoop-3.2.1/logs/,AVAILABLE}
2020-05-22 19:50:36,308 INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@7d9f158f{/static,file:///opt/hadoop-3.2.1/share/hadoop/hdfs/webapps/static/,AVAILABLE}
2020-05-22 19:50:36,536 INFO handler.ContextHandler: Started o.e.j.w.WebAppContext@6c372fe6{/,file:///opt/hadoop-3.2.1/share/hadoop/hdfs/webapps/datanode/,AVAILABLE}{/datanode}
2020-05-22 19:50:36,556 INFO server.AbstractConnector: Started ServerConnector@4fbe37eb{HTTP/1.1,[http/1.1]}{localhost:40135}
2020-05-22 19:50:36,557 INFO server.Server: Started @3690ms
2020-05-22 19:50:37,099 INFO web.DatanodeHttpServer: Listening HTTP traffic on /0.0.0.0:9864
2020-05-22 19:50:37,119 INFO util.JvmPauseMonitor: Starting JVM pause monitor
2020-05-22 19:50:37,135 INFO datanode.DataNode: dnUserName = root
2020-05-22 19:50:37,137 INFO datanode.DataNode: supergroup = supergroup
2020-05-22 19:50:37,300 INFO ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 1000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.2020-05-22 19:50:37,324 INFO ipc.Server: Starting Socket Reader #1 for port 9867
2020-05-22 19:50:37,999 INFO datanode.DataNode: Opened IPC server at /0.0.0.0:9867
2020-05-22 19:50:38,042 INFO datanode.DataNode: Refresh request received for nameservices: null
2020-05-22 19:50:39,732 WARN hdfs.DFSUtilClient: Namenode for null remains unresolved for ID null. Check your hdfs-site.xml file to ensure namenodes are configured properly.
2020-05-22 19:50:39,733 INFO datanode.DataNode: Starting BPOfferServices for nameservices: <default>
2020-05-22 19:50:39,748 INFO datanode.DataNode: Block pool <registering> (Datanode Uuid unassigned) service to primary-namenode:8020 starting to offer service
2020-05-22 19:50:39,764 INFO ipc.Server: IPC Server Responder: starting
2020-05-22 19:50:39,766 INFO ipc.Server: IPC Server listener on 9867: starting
2020-05-22 19:50:39,907 WARN datanode.DataNode: Problem connecting to server: primary-namenode:8020
2020-05-22 19:50:44,908 WARN datanode.DataNode: Problem connecting to server: primary-namenode:8020

[Alibm73-zz]

@0lucasmoura yes you can. just create a network and use it in all of your services.

services:
  namenode:
    ...[you configurations]
    networks:
      - hbase
    deploy:
      mode: replicated
      replicas: 1
      restart_policy:
        condition: on-failure
      placement:
        constraints:
          - node.hostname == [your node name]
  datanode:
    ...
    networks:
      - hbase
    deploy:
      mode: replicated
      replicas: 1
      restart_policy:
        condition: on-failure
      placement:
        constraints:
          - node.hostname == [your node name]

at the end type:

networks:
  hbase:
    external: true

[0lucasmoura]

Thanks for the answer! I have managed to do it by creating an overlay external network (defined outside the docker-compose.yml) and making the services connect to it!!

I believe that doing it inside the compose like you suggested would give the same results.