samples DevOps2.2 do not work

[lukicsl]

I started the samples in chapter. Deploying And Configuring Prometheus I first tried with my headless ubuntu server. Virtual machines up an running. The first sample : docker stack deploy -c stacks/prometheus.yml monitor I was not able to connect to http://$(docker-machine ip swarm-1):9090/config. Thought is was my proxy config to the headless server, did al sort of ssh port forwarding, etc, always getting connection refused. so I moved to my iMAC. Did all the needed setup regarding docker and docker machine. Getting the same result. After googling I managed to change the yml file to:

version: "3.2"

services:

  prometheus:
    image: prom/prometheus:${TAG:-latest}
    ports:
      - target: 9090
        published: 9090
        protocol: tcp
        mode: host

then was able to connect to config. the next example does not work eather in chapter Integrating Docker Flow Monitor With Docker Flow Proxy proxy is not able to connect to swarm listerner:

2018/12/27 18:23:41 Error: Fetching config from swarm listener failed: Get http://swarm-listener:8080/v1/docker-flow-swarm-listener/get-services: dial tcp 10.0.12.91:8080: connect: connection refused. Will retry in 5 seconds.
2018/12/27 18:23:46 Error: Fetching config from swarm listener failed: Get http://swarm-listener:8080/v1/docker-flow-swarm-listener/get-services: dial tcp 10.0.12.91:8080: connect: connection refused. Will retry in 5 seconds.

what is wrong here, my setup problem, or something changed with version?

my setup:

BigiMac:docker-flow-monitor slavisalukic$ docker version
Client:
 Version:           18.06.1-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        e68fc7a
 Built:             Tue Aug 21 17:21:31 2018
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.0
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.4
  Git commit:       4d60db4
  Built:            Wed Nov  7 00:52:55 2018
  OS/Arch:          linux/amd64
  Experimental:     false

BigiMac:docker-flow-monitor slavisalukic$ docker-machine version
docker-machine version 0.15.0, build b48dc28d

[vfarcic]

It seems that Overlay network inside your cluster does not work. The fact that it works when changing to host is an indication of that. The two errors you posted are also an indication of the same problem.

I'm a bit confused with the headless Ubuntu. Are you using Ubuntu as your laptop's OS (not a VM) and inside it creating a cluster using Docker Machine?

[lukicsl]

I have an older zotak where I installed a 18.4 LTS WS. I moved it after installation to a rack, with no monitor ( headless), had no luck to install any type of remote desktop. I installed the vms with your script and run the example. was not able to access http://$(docker-machine ip swarm-1):9090/config, getting connection refused, first with a proxy server on vm's host and not even with curl from local shell (host of vm's).

I then installed all prerequisites on my MAC, to verfy, an getting the issue.

What does mean "It seems that Overlay network inside your cluster does not work".

I could once again try it on my 3-node raspberry pi cluster, though would have to rebuild the images for arm.

[vfarcic]

Can you run any other application and confirm that it is accessible through a published port?

[lukicsl]

can you make a dirty sample?

[vfarcic]

I am only with my mobile right now. I should be online in a few days and can write it then.

[lukicsl]

version: "3"

services:

  grafana:
    image: grafana/grafana:${GRAFANA_TAG:-4.5.2}
    ports:
      - 3000:3000

does not work either open "http://$(docker-machine ip swarm-1):3000"

BigiMac:docker-flow-monitor slavisalukic$ docker service ls
ID                  NAME                 MODE                REPLICAS            IMAGE                    PORTS
jwx5x282mek2        grafana_grafana      replicated          1/1                 grafana/grafana:4.5.2    *:3000->3000/tcp
nqn9qgzlypxe        monitor_prometheus   replicated          1/1                 prom/prometheus:latest   *:9090->9090/tcp

BigiMac:docker-flow-monitor slavisalukic$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
df74e0f43645        bridge              bridge              local
52d0c9e124f4        docker_gwbridge     bridge              local
vuwwlg06ux6f        grafana_default     overlay             swarm
5e37abf13d9c        host                host                local
ik8hb5uawnuk        ingress             overlay             swarm
to5uz6cryhl7        monitor             overlay             swarm
f6sk0obopjxy        monitor_default     overlay             swarm
658ca743eed5        none                null                local
34937t2os14x        proxy               overlay             swarm

BigiMac:docker-flow-monitor slavisalukic$ docker stack ps grafana
ID                  NAME                IMAGE                   NODE                DESIRED STATE       CURRENT STATE           ERROR               PORTS
io9ydb5c9f7e        grafana_grafana.1   grafana/grafana:4.5.2   swarm-1             Running             Running 9 minutes ago                       

[lukicsl]

service ls shows the ports, stack ps not !?

[lukicsl]

I create following test setup:

docker network create -d overlay grafana_default

docker service create \
  --name  grafana \
  --publish target=3000,published=3000 \
  --replicas=1 \
  --network grafana_default \
  grafana/grafana:4.5.2

docker service create \
  --name  curl-test \
  --replicas=3 \
  --network grafana_default \
  alpine sleep 1000000000

ID=$(docker ps -q --filter label=com.docker.swarm.service.name=curl-test)
docker exec -it $ID apk add --update curl drill

test run:

BigiMac:$ docker exec -it $ID curl grafana:3000/login
curl: (7) Failed to connect to grafana port 3000: Connection refused
BigiMac:$ docker exec -it $ID drill grafana
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 53225
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 
;; QUESTION SECTION:
;; grafana. IN  A

;; ANSWER SECTION:
grafana.    600 IN  A   10.0.23.2

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 0 msec
;; SERVER: 127.0.0.11
;; WHEN: Fri Dec 28 12:05:49 2018
;; MSG SIZE  rcvd: 48

[lukicsl]

I run the same on my 3-node raspberry cluster and

it works

!?

it does not work on bot docker-machine setups (MAC and Ubuntu)

[lukicsl]

https://github.com/docker/machine/issues/4608

docker machine iso is the problem

for i in 1 2 3; do
    docker-machine create \
        swarm-$i --virtualbox-boot2docker-url "https://github.com/boot2docker/boot2docker/releases/download/v18.06.1-ce/boot2docker.iso"
done

fixed the problem