Post API call {{authentication}}{{environment}}/registrations failed with errno:7

[cetsupport]

Environment

Deploy the lab environment by sdk.

Following apps installed:

Proxy Assets Authentication Camunda Cms Discovery Formio Identities Services Core Admin Portal Ui

Symptom:

1. Get token for anonymous.
2. Then call POST api {{authentication}}{{environment}}/registrations get errno 7.

                        "errno": [
                            "integer",
                            7
                        ],
                        "error": [
                            "string",
                            "Failed to connect to api.identities.lab.ds port 80: Connection refused"
                        ],
                        "url": [
                            "string",
                            "http://api.identities.lab.ds/organizations"
                        ],

3. Direct call api: {{identities}}{{environment}}/organizations? will get correct result:

[
    {
        "id": 5,
        "uuid": "c9599da5-35a8-494c-9181-975d78be9694",
        "createdAt": "2017-12-08T14:57:10+00:00",
        "updatedAt": "2017-12-08T14:57:10+00:00",
        "deletedAt": null,
        "owner": "BusinessUnit",
        "ownerUuid": "8acf2710-eec3-4a7a-a665-33f77ba424fa",
        "personas": [
            "/app_dev.php/organization-personas/1b23a845-873a-4e74-b086-497f06d34930"
        ],
        "version": 1
    }
]

Question:

Any suggestion how to fix this issue?

[marioprudhomme]

Is the identities microservice enabled here?

If you login as the "system" user and query GET /individuals under the Identities/Individuals postman folder, do you get a response back?

What is the output of docker ps -a?

[cetsupport]

Thanks for the prompt reply @mario-digitalstate

Here is the docker ps -a output:

CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS                       PORTS                NAMES
8c1255aa2d08        dsportal_ng2admin            "/var/www/run-prod.sh"   About an hour ago   Up About an hour             80/tcp, 443/tcp      dsportal_ng2admin_1
acb953ea8861        dsadmin_ng2admin             "/var/www/run-prod.sh"   About an hour ago   Up About an hour             80/tcp, 443/tcp      dsadmin_ng2admin_1
ed82447a5928        nginx:1.11-alpine            "nginx -g 'daemon ..."   About an hour ago   Up About an hour             80/tcp, 443/tcp      dsservices_nginx_1
d9b1595bbbef        dsservices_php               "docker-php-entryp..."   About an hour ago   Up About an hour             9000/tcp             dsservices_php_1
945be0225177        mysql:5.7                    "docker-entrypoint..."   About an hour ago   Up About an hour (healthy)   3306/tcp             dsservices_db_1
cb42da8e6b23        nginx:1.11-alpine            "nginx -g 'daemon ..."   About an hour ago   Up About an hour             80/tcp, 443/tcp      dsidentities_nginx_1
7bd46ac75263        dsidentities_php             "docker-php-entryp..."   About an hour ago   Up About an hour             9000/tcp             dsidentities_php_1
add15f36efb9        mysql:5.7                    "docker-entrypoint..."   About an hour ago   Up About an hour (healthy)   3306/tcp             dsidentities_db_1
8798cd56ee33        dsformio_formio              "/bin/sh -c /srv/f..."   About an hour ago   Up About an hour             80/tcp               dsformio_formio_1
ed081c9cefe0        mongo:3.4.3                  "docker-entrypoint..."   About an hour ago   Up About an hour (healthy)   27017/tcp            dsformio_mongodb_1
d1c90a867d5a        nginx:1.11-alpine            "nginx -g 'daemon ..."   About an hour ago   Up About an hour             80/tcp, 443/tcp      dscms_nginx_1
db6b5f3c45be        dscms_php                    "docker-php-entryp..."   About an hour ago   Up About an hour             9000/tcp             dscms_php_1
d4444eb1af15        mysql:5.7                    "docker-entrypoint..."   About an hour ago   Up About an hour (healthy)   3306/tcp             dscms_db_1
2ea9cd0ff14a        dscamunda_camunda            "/usr/local/bin/co..."   About an hour ago   Up About an hour             8080/tcp             dscamunda_camunda_1
212875512aae        nginx:1.11-alpine            "nginx -g 'daemon ..."   About an hour ago   Up About an hour             80/tcp, 443/tcp      dsauthentication_nginx_1
cbef2cf0c8ae        dsauthentication_php         "docker-php-entryp..."   About an hour ago   Up About an hour             9000/tcp             dsauthentication_php_1
d39b766e5ea4        mysql:5.7                    "docker-entrypoint..."   About an hour ago   Up About an hour (healthy)   3306/tcp             dsauthentication_db_1
35ee9124f756        nginx:1.11-alpine            "nginx -g 'daemon ..."   2 hours ago         Up 2 hours                   80/tcp, 443/tcp      dsassets_nginx_1
8b0f4edd0755        dsassets_php                 "docker-php-entryp..."   2 hours ago         Up 2 hours                   9000/tcp             dsassets_php_1
9156273e630d        mysql:5.7                    "docker-entrypoint..."   2 hours ago         Up 2 hours (healthy)         3306/tcp             dsassets_db_1
0529ff004177        jwilder/nginx-proxy:alpine   "/app/docker-entry..."   2 hours ago         Up 2 hours                   0.0.0.0:80->80/tcp   ds_proxy_1
399f3c0b5e91        sdk_sdk                      "/bin/sh"                2 hours ago         Up 2 hours                                        sdk

If I login as the "system" user then query Get /individuals, the response are:

[
    {
        "id": 5,
        "uuid": "d0daa7e4-07d1-47e6-93f2-0629adaa3b49",
        "createdAt": "2017-12-08T14:57:10+00:00",
        "updatedAt": "2017-12-08T14:57:10+00:00",
        "deletedAt": null,
        "owner": "BusinessUnit",
        "ownerUuid": "b20a40d9-b95b-4462-b8f1-c7453b9b7067",
        "personas": [
            "/app_dev.php/individual-personas/df67d9d4-0dc1-4393-972a-a659b24ff117"
        ],
        "version": 1
    }
]

From above print, the identities microservice should work fine.

[cetsupport]

In docker dsauthentication_php_1, following log indicate service not accessible:

[2017-12-08 15:00:50] request.CRITICAL: Uncaught PHP Exception GuzzleHttp\Exception\ConnectException: "cURL error 7: Failed to connect to api.identities.lab.ds port 80: Connection refused (see http://curl.haxx.se/libcurl/c/libcurl-errors.html)" at /srv/api-platform/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php line 186 {"exception":"[object] (GuzzleHttp\\Exception\\ConnectException(code: 0): cURL error 7: Failed to connect to api.identities.lab.ds port 80: Connection refused (see http://curl.haxx.se/libcurl/c/libcurl-errors.html) at /srv/api-platform/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php:186)"} []

If try curl api.identities.lab.ds from docker dsauthentication_php_1, then will get connection refused:

curl: (7) Failed to connect to api.identities.lab.ds port 80: Connection refused

I think this is the same type of error (code 7).

The strange thing is that the host name ofapi.identities.lab.ds been resolved as 127.0.0.1 from host dsauthentication_php_1. Suppose no process listen on port 80 of server dsauthentication_php_1. Is that's the reason?

[marioprudhomme]

What version of Docker are you running on your host machine? Which virtualization? HyperV? What version of HyperV?

It feels like an issue between Docker networks/communication.

[cetsupport]

I run the docker version Docker version 17.09.0-ce, build afdb6d4 on top of windows 10.

HyperV version is:

Version
10.0.16299.15

[marioprudhomme]

Found the issue. We recently updated our internal DNS names to include the environment name (api.identities.lab.ds) and the docker-compose files still reflect the old names, making cross-microservice communications not work in lab mode. (Essentially, the Docker /etc/host file is out of date for each microservices)

Will create a patch in a moment.

Either do a full re-install after I've created the patch, or a quick fix to avoid doing a full install would be to:

1) Stop the authentication microservice container

cd /srv/authentication
docker-compose stop

2) Edit the docker-compose file (/srv/authentication/docker-compose.yml) to have .lab.ds, for example:

api.assets.lab.ds:${PROXY_HOST}
api.authentication.lab.ds:${PROXY_HOST}
...

3) Re-up the container:

docker-compose up -d

[marioprudhomme]

Lab DNS has been updated for each microservices.

[cetsupport]

Thanks, verified that the issue was resolved by the update. Suggest remove all app before reinstall, as many app's docker-compose.yml been updated.

[marioprudhomme]

Great. All of this lab DNS stuff is going away anyway once we have auto-discovery in place!