[INFRA-3156] Cert VPN, can't access https://cert.ci.jenkins.io

[jenkins-infra-bot]

I am working on a security fix for the core and in order to debug test failures for a CERT PR, I need access to cert.ci.jenkins.io.

So I followed the documentation: https://github.com/jenkins-infra/docker-openvpn/blob/main/README.md#howto-get-client-access (I work with Windows 10), my PR was accepted, so I was able to sync my fork then pull it to retrieve my certificat and create my own jenkins-infra.ovpn :

client
remote vpn.jenkins.io 443
ca "c:\\Users\\Kevin\\.cert\\jenkins-infra\\ca.crt"
cert "c:\\Users\\Kevin\\.cert\\jenkins-infra\\kevingrdj.crt"
key "c:\\Users\\Kevin\\.cert\\jenkins-infra\\kevingrdj.key"
auth-user-pass
dev tun
proto tcp
nobind
auth-nocache
script-security 2
persist-key
persist-tun

After an import on OpenVPN, I was able to connect, however I still can't access https://cert.ci.jenkins.io

Here are the OpenVPN logs:

2021-12-13 21:39:42 --cipher is not set. Previous OpenVPN version defaulted to BF-CBC as fallback when cipher negotiation failed in this case. If you need this fallback please add '--data-ciphers-fallback BF-CBC' to your configuration and/or add BF-CBC to --data-ciphers.
2021-12-13 21:39:42 OpenVPN 2.5.4 Windows-MSVC [SSL (OpenSSL)] [LZO] [LZ4] [PKCS11] [AEAD] built on Oct  5 2021
2021-12-13 21:39:42 Windows version 10.0 (Windows 10 or greater) 64bit
2021-12-13 21:39:42 library versions: OpenSSL 1.1.1k  25 Mar 2021, LZO 2.10
2021-12-13 21:39:51 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2021-12-13 21:39:51 TCP/UDP: Preserving recently used remote address: [AF_INET]104.209.251.202:443
2021-12-13 21:39:51 Attempting to establish TCP connection with [AF_INET]104.209.251.202:443 [nonblock]
2021-12-13 21:39:51 TCP connection established with [AF_INET]104.209.251.202:443
2021-12-13 21:39:51 TCP_CLIENT link local: (not bound)
2021-12-13 21:39:51 TCP_CLIENT link remote: [AF_INET]104.209.251.202:443
2021-12-13 21:39:52 WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1543', remote='link-mtu 1559'
2021-12-13 21:39:52 WARNING: 'keysize' is used inconsistently, local='keysize 128', remote='keysize 256'
2021-12-13 21:39:52 [vpn.jenkins.io] Peer Connection Initiated with [AF_INET]104.209.251.202:443
2021-12-13 21:39:53 open_tun
2021-12-13 21:39:53 tap-windows6 device [OpenVPN TAP-Windows6] opened
2021-12-13 21:39:53 Set TAP-Windows TUN subnet mode network/local/netmask = 10.8.0.0/10.8.0.19/255.255.254.0 [SUCCEEDED]
2021-12-13 21:39:53 Notified TAP-Windows driver to set a DHCP IP/netmask of 10.8.0.19/255.255.254.0 on interface {45ED140C-3877-42ED-B91F-48C092AE60CD} [DHCP-serv: 10.8.1.254, lease-time: 31536000]
2021-12-13 21:39:53 Successful ARP Flush on interface [7] {45ED140C-3877-42ED-B91F-48C092AE60CD}
2021-12-13 21:39:53 IPv4 MTU set to 1500 on interface 7 using service
2021-12-13 21:39:59 Initialization Sequence Completed

---

Originally reported by kevingrdj, imported from: Cert VPN, can't access https://cert.ci.jenkins.io

• assignee: kevingrdj
• status: Closed
• priority: Minor
• resolution: Fixed
• resolved: 2021-12-15T18:15:37+01:00
• imported: 2022/01/10

[jenkins-infra-bot]

hlemeur:

Kevin Guerroudj to be able to access cert.ci.jenkins.io with your VPN connection, you have to edit your ssh configuration file (`~/.ssh/config` on Linux/MacOS, should be in your user profile folder on Windows) and add this block:

Host cert.ci.jenkins.io
    # Private IP only reachable through the VPN network
    HostName 10.0.2.252
    # Your own username
    User 
    # Your private key associated to your username. SSH-agent can be used instead.
    IdentityFile private key>

Let me know here if it worked for you.

[jenkins-infra-bot]

wfollonier:

Hervé Le Meur, Damien Duportal no config entries in my case. No entry in C:\Windows\System32\drivers\etc\hosts. My gut feeling is that Kevin is lacking some permissions in the VPN server directly.

[jenkins-infra-bot]

wfollonier:

Less errors after added cipher AES-256-CBC
But still unreachable.

[jenkins-infra-bot]

JIRAUSER134225:

While trying to access with chrome to http://cert.ci.jenkins.io/, I have an error "DNS_PROBE_FINISHED_NXDOMAIN"

[jenkins-infra-bot]

dduportal:

First wave of analysis:

• Herve has access to cert-ci, which mean his configurations works (outside mine or Daniel/Wadeck's)
• The route on the OpenVPN server sounds good for Kevin and includes the correct Ip range: https://github.com/jenkins-infra/docker-openvpn/blob/main/cert/ccd/kevingrdj

[jenkins-infra-bot]

dduportal:

Second wave:

• DNS_PROBE_FINISHED_NXDOMAIN seems to be a Chrome-only error. Looks like that the browser cannot reach the domain/IP associated to the domain
• Kevin Guerroudj can you run a `dig cert.ci.jenkins.io` (or nslookup) to determine which IP is resolved to the domain on your machine?
• It feels like that this error means that the openVPN configuration is not working on Kevin's machine (either not connecting, or not applying the rule)

[jenkins-infra-bot]

dduportal:

• I confirm that SSH is not needed (I thought there would be a tunnel but it's not the case, fsorry for sending you on this wrong path). Beside, no need to overide the /etc/host entries (same, sorry for the false lead).
• If it works on Wadeck Follonier's machine as well, then it confirm there is an issue on the client side.

[jenkins-infra-bot]

wfollonier:

(not totally convinced it's client side but I am so an expert in VPN ... )
Do you need to enable/activate something in the VPN server configuration to allow Kevin to reach out to some parts of the network?

[jenkins-infra-bot]

dduportal:

Wadeck Follonierthe server-side configuration applies the following routing rules: https://github.com/jenkins-infra/docker-openvpn/blob/main/cert/ccd/kevingrdj as far as my VPN-fu knows (yeah I'm not a VPN expert either.. ).

Kevin Guerroudj do you have access to https://infra.ci.jenkins.io with the VPN connected?

• Yes: then there is an issue with routing to cert.ci.jenkins.io
• No: then there is a general VPN configuration issue.

Don't get me wrong folks: when I wrote "client side", I meant "for sure we never tried Windows and there is absolutely an issue that we never had before and we should fix on at least ou VPN doc"

[jenkins-infra-bot]

JIRAUSER134225:

No, I don't have access to https://infra.ci.jenkins.io with the VPN connected.

And for the nslookup, it's not very conclusive :

nslookup cert.ci.jenkins.io
Server: Unknown
Address: 192.168.1.254
*** No internal type record for both IPv4 and IPv6 Addresses (A+AAAA) available for cert.ci.jenkins.io

[jenkins-infra-bot]

dduportal:

[jenkins-infra-bot]

dduportal:

TL;DR;

• Kevin Guerroudj correctly configured his OpenVPN client and did good!
• As far as we know, Wadeck Follonier has a Windows machine where the VPN works: issues is not Windows
• We checked interactively with Kevin Guerroudj : his DNS resolution was not able to resolve either cert.ci.jenkins.io or infra.ci.jenkins.io, while the DNS records are publics.

=> We solved by switching to 9.9.9.9 (and also tested with 8.8.8.8) in complement to his current DNS, and everything worked!

[jenkins-infra-bot]

dduportal:

Resolving the issue. Kevin Guerroudj can you close the issue if it is ok for you (or feel free to re-open if you have an issue)