Is this a fully functional demo?

[macedemo]

I followed the instructions here: https://microservices-demo.github.io/docs/quickstart.html $git clone https://github.com/microservices-demo/microservices-demo $cd microservices-demo $docker-compose -f deploy/docker-compose/docker-compose.yml up -d

And the services started but when I did the load test 100% of the GET /login and POST /orders fail. Results below. When I used weave scope it does not show any communications between image weaveworksdemos/user and weaveworksdemos/user-db like I would expect per the architecture diagram. Also when I try to login via the webpage I get invalid login with the default passwords from : https://microservices-demo.github.io/docs/user-accounts.html. I ran this on Centos 3.10.0-693.5.2.el7.x86_64

Name # reqs # fails Avg Min Max | Median req/s

GET / 5 0(0.00%) 10 5 23 | 9 0.00 GET /basket.html 6 0(0.00%) 10 6 19 | 6 0.00 DELETE /cart 6 0(0.00%) 142 21 684 | 30 0.00 POST /cart 6 0(0.00%) 94 41 241 | 71 0.00 GET /catalogue 6 0(0.00%) 16 10 29 | 13 0.00 GET /category.html 5 0(0.00%) 7 5 16 | 5 0.00 GET /detail.html?id=03fef6ac-1896-4ce8-bd69-b798f85c6e0b 1 0(0.00%) 8 8 8 | 8 0.00 GET /detail.html?id=3395a43e-2d88-40de-b95f-e00e1502085b 1 0(0.00%) 5 5 5 | 5 0.00 GET /detail.html?id=510a0d7e-8e83-4193-b483-e27e09ddc34d 1 0(0.00%) 7 7 7 | 7 0.00 GET /detail.html?id=808a2de1-1aaa-4c25-a9b9-6612e8f29a38 2 0(0.00%) 5 5 5 | 5 0.00 GET /login 0 5(100.00%) 0 0 0 | 0 0.00 POST /orders 0 6(100.00%) 0 0 0 | 0 0.00

Total 39 11(28.21%) 0.00

Percentage of the requests completed within given times Name # reqs 50% 66% 75% 80% 90% 95% 98% 99% 100%

GET / 5 9 11 11 23 23 23 23 23 23 GET /basket.html 6 13 13 14 14 19 19 19 19 19 DELETE /cart 6 32 32 61 61 680 680 680 680 684 POST /cart 6 71 71 86 86 240 240 240 240 241 GET /catalogue 6 13 13 20 20 29 29 29 29 29 GET /category.html 5 5 8 8 16 16 16 16 16 16 GET /detail.html?id=03fef6ac-1896-4ce8-bd69-b798f85c6e0b 1 8 8 8 8 8 8 8 8 8 GET /detail.html?id=3395a43e-2d88-40de-b95f-e00e1502085b 1 5 5 5 5 5 5 5 5 5 GET /detail.html?id=510a0d7e-8e83-4193-b483-e27e09ddc34d 1 7 7 7 7 7 7 7 7 7 GET /detail.html?id=808a2de1-1aaa-4c25-a9b9-6612e8f29a38 2 5 5 5 5 5 5 5 5 5

Error report

occurences Error

---

6 POST /orders: "HTTPError(u'500 Server Error: Internal Server Error for url: http://localhost/orders',)" 6 GET /login: "HTTPError(u'401 Client Error: Unauthorized for url: http://localhost/login',)"

[pidster]

I can reproduce this.

[rndstr]

Thanks for reporting this @macedemo. I can't reproduce this on my end. All three logins work fine. But I got a flakey order test instead, which doesn't seem to be related to this (see https://github.com/microservices-demo/payment/pull/31 for fix).

Would you mind sharing logs of the following services?

docker logs dockercompose_orders_1
docker logs dockercompose_user_1

[macedemo]

Sure no problem.

orders.log user.log

[macedemo]

Some more info from my setup: $ docker --version Docker version 17.09.0-ce, build afdb6d4 $ docker-compose -version docker-compose version 1.17.0, build ac53b73 $uname -a Linux localhost.localdomain 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ lsb_release -a LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.4.1708 (Core) Release: 7.4.1708 Codename: Core

$docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES aa52463829b3 weaveworks/scope:1.6.5 "/home/weave/entry..." About an hour ago Up About an hour weavescope dc8727dc03e8 mongo "docker-entrypoint..." 3 days ago Up 2 hours 27017/tcp dockercompose_orders-db_1 87d9d5cca3e3 weaveworksdemos/user-db "docker-entrypoint..." 3 days ago Up 2 hours 27017/tcp dockercompose_user-db_1 66b8f913e4e3 rabbitmq:3.6.8 "docker-entrypoint..." 3 days ago Up 2 hours 4369/tcp, 5671-5672/tcp, 25672/tcp dockercompose_rabbitmq_1 694f5685618c weaveworksdemos/user "/user -port=80" 3 days ago Up 2 hours 80/tcp dockercompose_user_1 0a6d398bc434 weaveworksdemos/payment "/app -port=80" 3 days ago Up 2 hours 80/tcp dockercompose_payment_1 e02a31e71627 weaveworksdemos/shipping "/usr/local/bin/ja..." 3 days ago Up 2 hours dockercompose_shipping_1 bb8b2c173666 mongo "docker-entrypoint..." 3 days ago Up 2 hours 27017/tcp dockercompose_carts-db_1 d2dc6386f617 weaveworksdemos/carts "/usr/local/bin/ja..." 3 days ago Up 2 hours dockercompose_carts_1 ff5b91e1ac8b weaveworksdemos/orders "/usr/local/bin/ja..." 3 days ago Up 2 hours dockercompose_orders_1 d9c34e4300e5 weaveworksdemos/edge-router "traefik" 3 days ago Up 2 hours 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp dockercompose_edge-router_1 53a50bc9073c weaveworksdemos/catalogue "/app -port=80" 3 days ago Up 2 hours 80/tcp dockercompose_catalogue_1 013bd5ad26db weaveworksdemos/front-end "/usr/local/bin/np..." 3 days ago Up 2 hours 8079/tcp dockercompose_front-end_1 c0aca6fedeca weaveworksdemos/catalogue-db "docker-entrypoint..." 3 days ago Up 2 hours 3306/tcp dockercompose_catalogue-db_1 05e2c5fe5191 weaveworksdemos/queue-master "/usr/local/bin/ja..." 3 days ago Up 2 hours dockercompose_queue-master_1 76d321811f57 grafana/grafana "/run.sh" 12 days ago Up 2 hours 0.0.0.0:3000->3000/tcp dockercompose_grafana_1 0fbbe1c2b178 prom/prometheus "/bin/prometheus -..." 12 days ago Up 2 hours 0.0.0.0:9090->9090/tcp prometheus 7acf6edbc354 prom/alertmanager "/bin/alertmanager..." 12 days ago Up 2 hours 0.0.0.0:9093->9093/tcp dockercompose_alertmanager_1

[macedemo]

Looked at the user-db log also and found this error: chown: changing ownership of '/proc/1/fd/1': Permission denied chown: changing ownership of '/proc/1/fd/2': Permission denied

user-db.log

I can connect to the user-db/mongodb manually from mongodb client on host as per log.

[macedemo]

I have attempted to trouble shoot this problem a little on my own with my limited knowledge of Docker, microservices golang etc.

Something jumped out to me and I wonder if why rndstr could not recreate is because he ran and built it from the source and did not let the system pull the images from docker hub? The version of images I get on my system in some cases are fairly old and I don't think are synced with the master of each project on github. Thus following directions from here https://microservices-demo.github.io/docs/quickstart.html really would only work if dockerhub is sync with master right?

front-end latest d396582fc174 6 days ago 148MB weaveworksdemos/user latest 6cb76cfcb181 6 weeks ago 35.8MB weaveworksdemos/user-db latest eab8b6af1f11 6 weeks ago 676MB

weaveworksdemos/load-test latest 3ab3fd5cd04e 6 months ago 570MB weaveworksdemos/catalogue-db latest f42f8194e889 8 months ago 406MB weaveworksdemos/catalogue latest 0bd359b6d6e8 8 months ago 41.2MB

weaveworksdemos/payment latest 4f2c23055dcd 8 months ago 32.5MB weaveworksdemos/front-end latest b54402ef78a5 8 months ago 120MB weaveworksdemos/shipping latest 4fc533e8180a 8 months ago 199MB weaveworksdemos/carts latest c00473736118 8 months ago 198MB weaveworksdemos/orders latest 8275c5b9181b 8 months ago 198MB weaveworksdemos/queue-master latest 76f0de7a12ac 8 months ago 178MB weaveworksdemos/edge-router latest 584dac9095de 9 months ago 21.9MB

[macedemo]

I believe I have identified the issue. It's a visioning issue as per my last post. I have it working now but instead of using the default (i.e. latest label on an image) in the ./deploy/docker-compose/docker-compose.yml I went to the docker hub where the images are located and updated to the last one uploaded thus updated the docker-compose file as follows to use these images:

image: weaveworksdemos/front-end:master-f7775f58 image: weaveworksdemos/catalogue:master-9de2c86e-normal image: weaveworksdemos/catalogue-db:master-9de2c86e image: weaveworksdemos/carts:master-1c6d09fb image: weaveworksdemos/orders:master-f8988a3d image: weaveworksdemos/shipping:master-ba362a4c image: weaveworksdemos/queue-master:master-b8bcfc79 image: weaveworksdemos/payment:master-791d88d8 image: weaveworksdemos/user:master-a7ea6917 image: weaveworksdemos/user-db:master-a7ea6917 image: weaveworksdemos/load-test:master-57a88289

So I think someone needs to update the docker_compose file to pull the right versions of a working system or update docker hub and tag as latest the images that would work together.

Break---------- After a little more experimentation it looks like with all my experiments I might have drawn the wrong conclusion. It works now but it only works when I run the docker compose and my VM's NAT'ed address/interface is down. Then I can bring it backup to have internet access and it all works. So sounds like maybe some issue with networking in user/userdb service?

[macedemo]

Did a little more digging and I looked at the netstat of the user container when it does not work (have an internet connecting in the VM) and when it works (VM has no network enabled).

Seem t go out to google dns to resolve?

(not working) $ docker exec ec732f9bb542 netstat Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State udp 0 0 user:57098 google-public-dns-a.google.com:http ESTABLISHED udp 0 0 localhost:34028 127.0.0.11:domain ESTABLISHED udp 0 0 user:50598 172.16.194.2:domain ESTABLISHED Active UNIX domain sockets (w/o servers) Proto RefCnt Flags Type State I-Node Path

(when working) $ docker exec a4d2dcae7d3c netstat Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 user:46136 dockercompose_user-db_1.dockercompose_default:27017 ESTABLISHED tcp 0 0 user:http dockercompose_front-end_1.dockercompose_default:57930 TIME_WAIT tcp 0 0 user:http dockercompose_front-end_1.dockercompose_default:57922 TIME_WAIT udp 0 0 user:53417 8.8.8.8:http ESTABLISHED udp 0 0 user:58648 8.8.8.8:domain ESTABLISHED udp 0 0 user:38319 8.8.4.4:domain ESTABLISHED udp 0 0 user:51622 8.8.4.4:domain ESTABLISHED udp 0 0 user:36221 8.8.8.8:domain ESTABLISHED Active UNIX domain sockets (w/o servers) Proto RefCnt Flags Type State I-Node Path

[pidster]

@macedemo I've bumped the versions in the deploy/docker-* to the latest tags.

I only see errors on a 2016 MBP with Docker for Mac, if there's a race and the services are not all up before the user-sim runs.

[pidster]

Further testing: I don't understand this error:

GET /login: "HTTPError(u'401 Client Error: Unauthorized for url: http://localhost/login',)"

The others seem to be timing related, so I'm currently dismissing those - pending further information.

[macedemo]

@pidster That's basically a symptom I was seeing when not working is I could not login from the browser.

[pidster]

@macedemo That error also occurs when I run the load-test container - without any manual GUI interaction, which is why I don't understand it.

I'd expect Locust to account for any challenge-resend sequences, rather than reporting it as an error.