There was an error updating certificate "xxxx": Verification timed out

Andsup commented 5 months ago

FreePBX Version

FreePBX 17

Issue Description

Hi,

-nslookup/dig resolves correctly the url.

fwconsole cert --updateall

Processing: xxxx, Local IP: yyyy, Public IP: dns error Self test: trying http://xxxx/.freepbx-known/816d7aba7f5dfff89f4418d7950b3634 Self test: received 816d7aba7f5dfff89f4418d7950b3634 lechecker: Pest_Curl_Exec - Operation timed out after 30001 milliseconds with 0 bytes received

** lechecker: Pest_Curl_Exec - Operation timed out after 30001 milliseconds with 0 bytes received

** The LetsEncrypt Service is listening on port disabled. Using a custom port other than 80 is not officially supported.

  The LetsEncrypt servers only send challenge queries to port 80. Certificate requests will fail unless your network redirects incoming port 80 requests to port disabled.

Certificate named "default" is valid There was an error updating certificate "xxxx": Verification timed out

Operating Environment

debian 12.5 all freepbx modules up to date in edge mode. letsencrypt protocol managed by Responsive LetsEncrypt Rules.

Relevant log output

In the Apache log file : xxxx 80 yyyy - - [09/Jun/2024:07:59:11 +0200] "GET /.well-known/acme-challenge/3NtMndJgDMS9QlUUaCfrtCqorM-fRzx5vg-OhV80V8w HTTP/1.1" 200 320 "-" "Wget/ (Red Hat modified)" xxxx 80 yyyy - - [09/Jun/2024:07:59:52 +0200] "GET /.freepbx-known/816d7aba7f5dfff89f4418d7950b3634 HTTP/1.1" 200 258 "-" "-" xxxx 80 yyyy - - [09/Jun/2024:08:00:24 +0200] "GET /.well-known/acme-challenge/duMoDm3Ih-YaqRwZe87LaVHIfroD9AO2VYvjYLyx4Vw HTTP/1.1" 200 320 "-" "Wget/ (Red Hat modified)

ramarajan222 commented 5 months ago

Hi @Andsup ,

The Let's Encrypt certificate update is working fine on our test system. Can you confirm that port 80 is set up for Let's Encrypt? If not, set port 80 to Let's Encrypt and give it a try. Thanks.

Andsup commented 5 months ago

This machine does not have Sysadmin Pro activated: LetsEncrypt port change not allowed

This acme-challenge request is well received on port 80:

xxxx 80 yyyy - - [10/Jun/2024:08:10:53 +0200] "GET /.well-known/acme-challenge/uro9dYN7eC3sjLHeGi5ctVXSjv34kXhO0tAWnxZ_TpA HTTP/1.1" 200 320 "-" "Wget/ (Red Hat modified)"

ramarajan222 commented 5 months ago

Hi @Andsup Let's Encrypt certificates will work only with Port 80. So first, change the admin port to something other than 80, then try to set Let's Encrypt to use Port 80.

Andsup commented 5 months ago

Successfully updated certificate.

But : in cert admin :

In the browser after a restart of freepbx and the browser :

Now firewall is not working as expected : http web requests allowed !

Reversed back to admin on port 80.

Is that a standard way of working ? Why not set as default config ?

ramarajan222 commented 5 months ago

Yes. Freepbx V16 also works the same way so its expected behaviour.

Andsup commented 5 months ago

What about the validity ?

No ok from my point of view.

jasonblewis commented 5 months ago

Turning off the firewall during the certificate update process allowed me to renew the certificate https://github.com/FreePBX/issue-tracker/issues/77

Once I did that I had to make sure Apache was still using the correct certificate and restart it

iMiMx commented 4 months ago

In my case, it doesn't seem that FreePBX 17 is adding port 80 to the ipset - I see the following firewall rules:

root@freepbx:/etc/cron.daily# iptables-save |grep lefilter
:lefilter - [0:0]
-A fpbxfirewall -p tcp -m set --match-set lefilter dst -j lefilter
-A lefilter -m state --state NEW -j CONNMARK --set-xmark 0x20/0xffffffff
-A lefilter -m state --state NEW -j ACCEPT
-A lefilter -m string --string "GET /.well-known/acme-challenge/" --algo kmp -j ACCEPT
-A lefilter -m string --string "GET /.freepbx-known/" --algo kmp -j ACCEPT
-A lefilter -j RETURN

But the ipset lefilter is never populated with port 80 when running a 'fwconsole cert --updateall --force'

root@freepbx:~# ipset --list
Name: lefilter
Type: bitmap:port
Revision: 3
Header: range 80-65535 timeout 60
Size in memory: 531904
References: 2
Number of entries: 0
Members:

If I fire-off the update and then add:

ipset add lefilter 80

... then it works.

ramarajan222 commented 4 months ago

Hi @iMiMx ,

we were unable to replicate your issue on our test system, and the LE certificate update is working fine with the firewall enabled.

` root@uc-test:/home# fwconsole cert --updateall --force Forced update enabled !!! Processing: freepbx17.test.com, Local IP: XXX.XXX.XXX.XXX, Public IP: XXX.XXX.XXX.XXX Self test: trying http://freepbx17.test.com/.freepbx-known/ae355f4e4444a68f84a140284751e37e Self test: received ae355f4e4444a68f84a140284751e37e Successfully updated certificate named "freepbx17.test.com"

`

Also while running the certificate update, we saw port 80 was populated on the ipset.

` root@uc-test:~# ipset --list Name: lefilter Type: bitmap:port Revision: 3 Header: range 80-65535 timeout 60 Size in memory: 531904 References: 2 Number of entries: 1 Members: 80 timeout 58

` My test systems module versions are below

`

Please make sure your system is updated with the latest modules and port 80 is set to LE. Also, while running the certificate update command, you can check the cron log with tail -f /var/log/cron.log to confirm whether the "updateipset" hook is running or not. If you see any error logs, then raise a new issue with the log.

iMiMx commented 4 months ago

Ah ha - so I need to change the Insecure port of the admin from port 80, to something else, so that it can then be assigned exclusively to LetsEncrypt? If that is indeed documented somewhere, I didn't find it :)

... this then seems to work, after changing Admin http port to 8080. Thanks.

EDIT: Looks like I didn't read through the posts above thoroughly enough, where it is clarified to change HTTP to something else to then assign to LE, my apologies.

FreePBX / issue-tracker