search

OpenConext / OpenConext-deploy

Ansible-based deployment automation for the OpenConext platform
Apache License 2.0
12 stars 21 forks source link

Error in oidcng when playbook works #292

Closed Melkin1968 closed 3 years ago

Melkin1968 commented 4 years ago

Hi, Ive tried to make deploy OpneConext on VM and carefully made all recommendations signed in your deployment documentation. However, after the start of the provision script, it finished with error. After examination of logs, Ive found that the issue happened in the time of start oidcng.

Could you please help me to resolve the issue?

I`ve included logfile for oidcng. Thank you oidcng.zip

tvdijen commented 4 years ago

It seems like you have a syntax error in one of the json-files

Melkin1968 commented 4 years ago

Dear Tim,

Thank you a lot for your kind help. I agree with your suggestion however, I don`t aware which files may contain the error. Can you suggest which files may contain the error? I have only set files prepared by your manual: drwxrwxr-x. 3 centos centos 25 Sep 18 22:41 environments-external drwxrwxr-x. 2 centos centos 36 Sep 18 22:34 templates drwxrwxr-x. 2 centos centos 105 Sep 18 22:34 tests drwxrwxr-x. 2 centos centos 179 Sep 18 22:34 scripts drwxrwxr-x. 3 centos centos 27 Sep 18 22:34 tasks drwxrwxr-x. 64 centos centos 4096 Sep 18 22:34 roles -rw-rw-r--. 1 centos centos 133 Sep 18 22:34 playbook_haproxy.yml -rwxrwxr-x. 1 centos centos 3927 Sep 18 22:34 prep-env -rwxrwxr-x. 1 centos centos 2923 Sep 18 22:34 provision -rw-rw-r--. 1 centos centos 5529 Sep 18 22:34 provision.yml drwxrwxr-x. 2 centos centos 91 Sep 18 22:34 group_vars drwxrwxr-x. 2 centos centos 58 Sep 18 22:34 library drwxrwxr-x. 4 centos centos 32 Sep 18 22:34 environments -rw-rw-r--. 1 centos centos 7765 Sep 18 22:34 README.md -rw-rw-r--. 1 centos centos 1762 Sep 18 22:34 Vagrantfile -rw-rw-r--. 1 centos centos 3893 Sep 18 22:34 DEVELOPMENT.md -rw-rw-r--. 1 centos centos 10174 Sep 18 22:34 LICENSE

Thank you in advance, Yevhenii

Yevhenii Preobrazhenskyi *Technical department*, ISP URAN Tel: +380 (44) 454-9816 | Mobile: +380 (63) 720-1743 eugene.melkin@gmail.com eugene.melkin@gmail.com | www.uran.ua http://www.uran.ua

сб, 19 сент. 2020 г. в 17:00, Tim van Dijen notifications@github.com:

It seems like you have a syntax error in one of the json-files

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/OpenConext/OpenConext-deploy/issues/292#issuecomment-695217069, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABWKM6EO5CGMAIPULKGVXW3SGS2RNANCNFSM4RS7G47A .

tvdijen commented 4 years ago

I was thinking specifically about the json-templates in roles/oidcng/templates... What tag-version are you using, because scrolling through the commit log, this seems like a smoking gun..

BTW, you may get a faster response from the community on the OpenConext Slack-channel or the mailing list..

Melkin1968 commented 4 years ago

I use : [centos@peanoconext OpenConext-deploy]$ git describe --tag 322.1-41-g0a6d815

Thank you for your suggestion, I try to ask the community.

Melkin1968 commented 4 years ago

I've found the file which is the cause of the issue. It's /opt/oidcng/secret_keyset.json which is simply empty. However, I don`t know what content may be in it. How the file should be created and where it should be created in ansible script.

thijskh commented 4 years ago

There’s some details about this file and a Java program to generate it at https://github.com/OpenConext/OpenConext-oidcng/blob/master/README.md#endpoints The deploy should not be failing on this I think, something to look in to.

quartje commented 4 years ago

Maybe java was not installed on the machine where you ran the prep-env script? After installing java, you can create a keyset with this command:

scripts/gen_tink_keyset_oidc.sh output.keyset

The file output.keyset now contains the keyset you need to put into your secrets file (located in environments-external/$YOURENV/secrets/$YOURENV.yml under the key: oidcng_secret_keyset: Please refer to this file for the correct syntax: https://github.com/OpenConext/OpenConext-deploy/blob/master/environments/vm/secrets/vm.yml#L270

Melkin1968 commented 4 years ago

Thanks a lot for all who helped me!

Ive found there is a template roles/oidcng/templates/secret_keyset.json.j2 which contains simple '{{ oidcng_secret_keyset }}' however I cant find any procedure which should fulfil the template. So, each time when I start the provision script I get the empty file secret_keyset.json. OK. I`ve corrected it and I move further in deployment completion.

However, I could not finish it. Now I am stuck with this error: TASK [../../roles/vm_only_provision_manage_eb : push metadata eb] ***** fatal: [212.111.212.42]: FAILED! => {"cache_control": "no-cache, no-store, max-age=0, must-revalidate", "changed": false, "connection": "close", "content": "{\"timestamp\":\"2020-09-20T15:32:40.111+00:00\",\"status\":500,\"error\":\"Internal Server Error\",\"message\":\"\",\"path\":\"/internal/push\",\"exception\":\"org.springframework.web.client.ResourceAccessException\"}", "content_type": "application/json", "date": "Sun, 20 Sep 2020 15:32:40 GMT", "elapsed": 15, "expires": "0", "json": {"error": "Internal Server Error", "exception": "org.springframework.web.client.ResourceAccessException", "message": "", "path": "/internal/push", "status": 500, "timestamp": "2020-09-20T15:32:40.111+00:00"}, "msg": "Status code was 500 and not [200]: HTTP Error 500: ", "pragma": "no-cache", "redirected": false, "status": 500, "transfer_encoding": "chunked", "url": "http://127.0.0.1:9393/internal/push", "x_content_type_options": "nosniff", "x_frame_options": "DENY", "x_xss_protection": "1; mode=block"}

NO MORE HOSTS LEFT ****

PLAY RECAP **** 212.111.212.42 : ok=400 changed=1 unreachable=0 failed=1 skipped=205 rescued=0 ignored=0

Could you please share some ideas with me - where I have to look for the cause of a new issue?

Best regards

quartje commented 4 years ago

We just fixed our documentation this morning regarding this issue. The problem is here that the loadbalancer is listening on localhost. It's listening on 127.0.0.1 and 127.0.0.2. To fix the provisioning, you could make sure that the engine-api. on the target machine resolves to 127.0.0.2, and rerun the deployment (if you add --tags vm_only_provision_manage_eb to your ./provision command it will only run the last task).

If you want to fix it and give your apps an external ip, please refer to the changed documentation on the wiki (see https://github.com/OpenConext/OpenConext-deploy/wiki/Installation-steps-to-deploy-OpenConext-on-a-single-system-other-than-the-Vagrant-VM-centOS7#prepare-for-external-connectivity).

Melkin1968 commented 4 years ago

No luck, unfortunately: TASK [../../roles/vm_only_provision_manage_eb : push metadata eb] ***** fatal: [212.111.212.42]: FAILED! => {"cache_control": "no-cache, no-store, max-age=0, must-revalidate", "changed": false, "connection": "close", "content": "{\"timestamp\":\"2020-09-21T17:39:15.206+00:00\",\"status\":500,\"error\":\"Internal Server Error\",\"message\":\"\",\"path\":\"/internal/push\",\"exception\":\"org.springframework.web.client.HttpServerErrorException.ServiceUnavailable\"}", "content_type": "application/json", "date": "Mon, 21 Sep 2020 17:39:15 GMT", "elapsed": 8, "expires": "0", "json": {"error": "Internal Server Error", "exception": "org.springframework.web.client.HttpServerErrorException.ServiceUnavailable", "message": "", "path": "/internal/push", "status": 500, "timestamp": "2020-09-21T17:39:15.206+00:00"}, "msg": "Status code was 500 and not [200]: HTTP Error 500: ", "pragma": "no-cache", "redirected": false, "status": 500, "transfer_encoding": "chunked", "url": "http://127.0.0.1:9393/internal/push", "x_content_type_options": "nosniff", "x_frame_options": "DENY", "x_xss_protection": "1; mode=block"}

NO MORE HOSTS LEFT ****

PLAY RECAP **** 212.111.212.42 : ok=400 changed=0 unreachable=0 failed=1 skipped=205 rescued=0 ignored=0

I`ve got all your recommendations carefully. I have only 1 public IP: environments-external/peanoconext/group_vars/peanoconext.yml

haproxy_applications:

haproxy_backend_tls: False

haproxy_sni_ip: ipv4: 212.111.212.42 ipv6: "::1" certs:

haproxy_sni_ip_restricted: ipv4: 127.0.0.2 ipv6: "::1" certs:

Public IP names declared in DNS zone. Restricted IP names declared in /etc/hosts on the target: 127.0.0.2 authz-admin.peanoconext.uran.ua engine-api.peanoconext.uran.ua manage.peanoconext.uran.ua pdp.peanoconext.uran.ua 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

So, I don`t know where I have to look into next.

I`ll be appreciated for your help.

quartje commented 4 years ago

I think that if you remove the names (apart from localhost stuff) from /etc/hosts, and add the restricted apps to the DNS as well (so have engine-api.peanoconext.uran.ua and others point to 212.111.212.42) it should work

quartje commented 3 years ago

I am closing this issue. If the problems persist, don't hesitate to open another issue.