search

atkrad / wait4x

Wait4X allows you to wait for a port or a service to enter the requested state.
https://wait4x.dev
Apache License 2.0
266 stars 18 forks source link

Support self-signed TLS certiticate on HTTP component #95

Open mortymacs opened 2 years ago

lotyp commented 1 year ago

try mkcert and it will be accepted as valid in your system https://github.com/FiloSottile/mkcert

mortymacs commented 1 year ago

Hi @lotyp Thanks for the suggestion, but the idea is to use the already prepared certificates. For example, some APIs need certificate files prepared by the company and it's not public certificates. This issue is about those types of certificates.

sbaeurle commented 1 year ago

@mortymacs would an approach similar to https://forfuncsake.github.io/post/2017/08/trust-extra-ca-cert-in-go-app/ work? maybe using SystemCertPool instead of NewCertPool?

mortymacs commented 1 year ago

@sbaeurle I was thinking the same to use SystemCertPool by supporting additional CA file, but maybe it should support these 3 type of inputs like helm install command:

--ca-file
--cert-file
--key-file

What's your idea?

sbaeurle commented 1 year ago

Probably a good approach, what kind of formats would you expect for each argument? pem, crt and cem?

mortymacs commented 1 year ago

We're looking for something like: this https://gist.github.com/michaljemala/d6f4e01c4834bf47a9c4 What's your opinion?

sbaeurle commented 1 year ago

Looks like a good approach! The only open question on my side would be how to handle the system root store when we provide a custom certificate. Do we just append to the "internal" root store or do we override it and only accept our root cert?

mortymacs commented 1 year ago

It should be append in the system root store by using SystemCertPool.

mortymacs commented 1 year ago

So, would you like to send a PR or I send it?

sbaeurle commented 1 year ago

I can take a look at it. I'm probably quite booked this week, but next week looks promising and I will try to adjust the certificate handling then :)

mortymacs commented 1 year ago

@sbaeurle awesome!

vbnetadmin commented 1 year ago

Today tried to use wait4x on my local domains, like https://router.docker, or https://auth.wod.docker And I don't understand, is this problem, because I've installed custom certs with mkcert on my machine, and wait4x can't see them?

image image

But, in browser, I can open these domains:

image

I'm using traefik locally, with mkcert

I have macos Ventura 13.0.1

Interesting, that if I try, shipped curl with macos, it works and can access domains:

image

I'm wondering, is this related to custom SSL certs ?

Currently, can't use wait4x to ping services using http

sbaeurle commented 1 year ago

@vbnetadmin please add -l debug to the wait4x command, so we can verify if the incorrect ssl validation is indeed the problem

PS. logs in text format are better than screenshots, so we can copy and compare it more easily

vbnetadmin commented 1 year ago

Output:

Wait4X with debug

➜ wait4x http https://router.docker -l debug
2022-11-29T12:43:09+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:09+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:10+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:10+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:11+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:11+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:12+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:12+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:13+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:13+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:14+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:14+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:15+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:15+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:16+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:16+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:17+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:17+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:18+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:18+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host

Ping:

ping router.docker
PING router.docker (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.051 ms
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.104 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.260 ms
64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.311 ms

Brew curl:

➜ curl -L router.docker
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

macOS default curl:

➜ /usr/bin/curl -L https://router.docker
<!DOCTYPE html><html><head><title>Traefik</title><meta charset=utf-8><meta name=description content="Traefik UI"><meta name=format-detection content="telephone=no"><meta name=msapplication-tap-highlight content=no><meta name=viewport content="user-scalable=no,initial-scale=1,maximum-scale=1,minimum-scale=1,width=device-width"><link rel=icon type=image/png href=statics/app-logo-128x128.png><link rel=icon type=image/png sizes=16x16 href=statics/icons/favicon-16x16.png><link rel=icon type=image/png sizes=32x32 href=statics/icons/favicon-32x32.png><link rel=icon type=image/png sizes=96x96 href=statics/icons/favicon-96x96.png><link rel=icon type=image/ico href=statics/icons/favicon.ico><link rel=apple-touch-icon href=statics/icons/apple-icon-152x152.png><link rel=apple-touch-icon sizes=152x152 href=statics/icons/apple-icon-152x152.png><link rel=apple-touch-icon sizes=167x167 href=statics/icons/apple-icon-167x167.png><link rel=apple-touch-icon sizes=180x180 href=statics/icons/apple-icon-180x180.png><link href=css/041f4ca8.f6e3d404.css rel=prefetch><link href=css/05759188.7a9c0db7.css rel=prefetch><link href=css/09354cb8.0e433876.css rel=prefetch><link href=css/09995f48.60caa8f0.css rel=prefetch><link href=css/0c428e32.ed2ba60f.css rel=prefetch><link href=css/18c6e965.5d8876c8.css rel=prefetch><link href=css/3e3ce03c.9b22e80f.css rel=prefetch><link href=css/46fd955e.9b22e80f.css rel=prefetch><link href=css/491024e9.9b22e80f.css rel=prefetch><link href=css/4f0a01e4.f8382310.css rel=prefetch><link href=css/52875482.9b22e80f.css rel=prefetch><link href=css/54b4dbdf.6e23d447.css rel=prefetch><link href=css/6d73c73c.9b22e80f.css rel=prefetch><link href=css/77d911b4.9b22e80f.css rel=prefetch><link href=css/797e4f23.9b22e80f.css rel=prefetch><link href=css/b34404c8.9b22e80f.css rel=prefetch><link href=js/041f4ca8.4f70d06d.js rel=prefetch><link href=js/05759188.f1376142.js rel=prefetch><link href=js/09354cb8.2e39e534.js rel=prefetch><link href=js/09995f48.a140ed0a.js rel=prefetch><link href=js/0c428e32.92dcacf8.js rel=prefetch><link href=js/18c6e965.5dc22ece.js rel=prefetch><link href=js/2d21e8fd.c00ac0e6.js rel=prefetch><link href=js/3e3ce03c.293f4729.js rel=prefetch><link href=js/46fd955e.343a8488.js rel=prefetch><link href=js/491024e9.5ba4955c.js rel=prefetch><link href=js/4f0a01e4.e6772caf.js rel=prefetch><link href=js/52875482.65e645a6.js rel=prefetch><link href=js/54b4dbdf.915c5773.js rel=prefetch><link href=js/6d73c73c.d8a27bbb.js rel=prefetch><link href=js/77d911b4.8d5002b7.js rel=prefetch><link href=js/797e4f23.771f63ae.js rel=prefetch><link href=js/b34404c8.7a4b9be8.js rel=prefetch><link href=css/app.3d2d5994.css rel=preload as=style><link href=js/app.45ca101c.js rel=preload as=script><link href=js/runtime.db7dd4a3.js rel=preload as=script><link href=js/vendor.f60bcb86.js rel=preload as=script><link href=css/app.3d2d5994.css rel=stylesheet></head><body><div id=q-app></div><script type=text/javascript src=js/app.45ca101c.js></script><script type=text/javascript src=js/runtime.db7dd4a3.js></script><script type=text/javascript src=js/vendor.f60bcb86.js></script></body></html>%
mortymacs commented 1 year ago

Hi Patrick,

This is the DNS problem. Wait4x tries to lookup the domain via 100.64.0.2 DNS server which is depends on your system configuration. Would you please try with dig router.docker @100.64.0.2 and let us know the result?

vbnetadmin commented 1 year ago

@mortymacs

➜ dig router.docker @100.64.0.2

; <<>> DiG 9.10.6 <<>> router.docker @100.64.0.2 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11785 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION: ;router.docker. IN A

;; Query time: 35 msec ;; SERVER: 100.64.0.2#53(100.64.0.2) ;; WHEN: Tue Nov 29 13:02:07 EET 2022 ;; MSG SIZE rcvd: 31

vbnetadmin commented 1 year ago

I'm using dnsmasq on my mac, and my ansible role https://github.com/wayofdev/ansible-role-dnsmasq to setup dns on mac machines.

➜ cat /etc/resolver/docker nameserver 127.0.0.1

➜ pwd /opt/homebrew/etc

➜ cat dnsmasq.conf address=/docker/127.0.0.1 address=/mac/127.0.0.1

mortymacs commented 1 year ago

@mortymacs

➜ dig router.docker @100.64.0.2

; <<>> DiG 9.10.6 <<>> router.docker @100.64.0.2 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11785 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION: ;router.docker. IN A

;; Query time: 35 msec ;; SERVER: 100.64.0.2#53(100.64.0.2) ;; WHEN: Tue Nov 29 13:02:07 EET 2022 ;; MSG SIZE rcvd: 31

So, as it says, it couldn't fine anything.

I'm using dnsmasq on my mac, and my ansible role https://github.com/wayofdev/ansible-role-dnsmasq to setup dns on mac machines.

➜ cat /etc/resolver/docker nameserver 127.0.0.1

➜ pwd /opt/homebrew/etc

➜ cat dnsmasq.conf address=/docker/127.0.0.1 address=/mac/127.0.0.1

Well, this issue is not related to Wait4X, but, in order to address your issue, we need to know how do you run Wait4X? by Docker or binary on your local machine?

vbnetadmin commented 1 year ago

@mortymacs tried locally using binary and through docker (in separate image)

➜ which wait4x /usr/local/bin/wait4x

Quick result, using docker image:

docker run --rm --name='wait4x' \
    atkrad/wait4x:latest http https://router.docker -l debug
2022-11-29T12:13:32Z INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:13:32Z DBG Get "https://router.docker": dial tcp 127.0.0.1:443: connect: connection refused
2022-11-29T12:13:33Z INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:13:33Z DBG Get "https://router.docker": dial tcp 127.0.0.1:443: connect: connection refused
2022-11-29T12:13:35Z INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:13:35Z DBG Get "https://router.docker": dial tcp 127.0.0.1:443: connect: connection refused
2022-11-29T12:13:36Z INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:13:36Z DBG Get "https://router.docker": dial tcp 127.0.0.1:443: connect: connection refused

But, as you can see, using docker image for wait4x, it shows proper IP address

As a workaround, temporarily I can compile wait4x inside keycloak container (where I need to check for readiness)

Update:

Ok, this looks more promising, by adding flag --net=host I get this result:

➜ docker run --rm --net=host --name='wait4x' \ atkrad/wait4x:latest http http://router.docker -l debug 2022-11-29T12:19:35Z INF [HTTP] Checking the http://router.docker ... 2022-11-29T12:19:35Z DBG Get "https://router.docker/": x509: certificate signed by unknown authority 2022-11-29T12:19:36Z INF [HTTP] Checking the http://router.docker ... 2022-11-29T12:19:36Z DBG Get "https://router.docker/": x509: certificate signed by unknown authority 2022-11-29T12:19:37Z INF [HTTP] Checking the http://router.docker ... 2022-11-29T12:19:37Z DBG Get "https://router.docker/": x509: certificate signed by unknown authority 2022-11-29T12:19:38Z INF [HTTP] Checking the http://router.docker ... 2022-11-29T12:19:38Z DBG Get "https://router.docker/": x509: certificate signed by unknown authority 2022-11-29T12:19:39Z INF [HTTP] Checking the http://router.docker ... 2022-11-29T12:19:39Z DBG Get "https://router.docker/": x509: certificate signed by unknown authority

vbnetadmin commented 1 year ago

Temporary, working solution

docker-compose.yaml:

...
  keycloak:
    image: quay.io/keycloak/keycloak:20.0.1
    container_name: ${COMPOSE_PROJECT_NAME_SAFE}_keycloak
    restart: on-failure
    ports:
      - "8765:8080"
...

Makefile:

_kc-wait:
    $(DOCKER) run --rm --net=host --name='wait4x' atkrad/wait4x:latest \
        http $(shell docker inspect --format 'http://127.0.0.1:{{ (index (index .NetworkSettings.Ports "8080/tcp") 0).HostPort }}' wod_keycloak) \
        -l debug \
        -t 1m
.PHONY: _kc-wait

Result:

➜ make _kc-wait docker run --rm --net=host --name='wait4x' atkrad/wait4x:latest \ http http://127.0.0.1:8765 \ -l debug \ -t 1m 2022-11-29T12:59:19Z INF [HTTP] Checking the http://127.0.0.1:8765 ...

Btw, wait4x does not work, if container port of target isn't statically exposed:

    ports:
      - "8080"

➜ make _kc-wait docker run --rm --net=host --name='wait4x' atkrad/wait4x:latest \ http http://127.0.0.1:64584 \ -l debug \ -t 1m 2022-11-29T13:02:32Z INF [HTTP] Checking the http://127.0.0.1:64584 ... 2022-11-29T13:02:32Z DBG Get "http://127.0.0.1:64584": dial tcp 127.0.0.1:64584: connect: connection refused 2022-11-29T13:02:33Z INF [HTTP] Checking the http://127.0.0.1:64584 ... 2022-11-29T13:02:33Z DBG Get "http://127.0.0.1:64584": dial tcp 127.0.0.1:64584: connect: connection refused 2022-11-29T13:02:34Z INF [HTTP] Checking the http://127.0.0.1:64584 ... 2022-11-29T13:02:34Z DBG Get "http://127.0.0.1:64584": dial tcp 127.0.0.1:64584: connect: connection refused 2022-11-29T13:02:35Z INF [HTTP] Checking the http://127.0.0.1:64584 ... 2022-11-29T13:02:35Z DBG Get "http://127.0.0.1:64584": dial tcp 127.0.0.1:64584: connect: connection refused 2022-11-29T13:02:36Z INF [HTTP] Checking the http://127.0.0.1:64584 ... 2022-11-29T13:02:36Z DBG Get "http://127.0.0.1:64584": dial tcp 127.0.0.1:64584: connect: connection refused

docker compose ps NAME COMMAND SERVICE STATUS PORTS wod_keycloak "/opt/keycloak/bin/k…" keycloak running (healthy) 8443/tcp, 0.0.0.0:64584->8080/tcp

sbaeurle commented 1 year ago

@vbnetadmin I'm not sure if you understand networking and docker correctly (see https://docs.docker.com/network/). Your first approach using a docker container and the localhost address may never work per default (since each container usually has it's own virtual network adapter).

If instead of 127.0.0.1 you would use the actual address of the service/host you want to check (either the IP address or the compose service name) then wait4x would be able to find a route to the service and verify it.

atkrad commented 1 year ago

@vbnetadmin, If you can't fix the problem, you can use the --insecure-skip-tls-verify flag to skip TLS verification.