search

ragibkl / adblock-dns-server

Adblock DNS Server powered by Bancuh DNS and dnsdist-acme
https://bancuh.com/
MIT License
65 stars 14 forks source link

SSL Cert not updating #148

Closed ragibkl closed 2 years ago

ragibkl commented 2 years ago

I'm seeing the following logs. It might cause DOH and DOT to fail. For now, I manually restart the servers each month.

dnsdist_1       | Certificate not yet due for renewal
dnsdist_1       | 
dnsdist_1       | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
dnsdist_1       | Certificate not yet due for renewal; no action taken.
dnsdist_1       | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
dnsdist_1       | updating ssl cert complete
dnsdist_1       | reloading ssl cert
dnsdist_1       | Got control connection from 127.0.0.1:57036
dnsdist_1       | 140157762947896:error:0200100D:system library:fopen:Permission denied:crypto/bio/bss_file.c:288:fopen('/etc/letsencrypt/live/sg-dns1.bancuh.com/fullchain.pem','r')
dnsdist_1       | 140157762947896:error:20074002:BIO routines:file_ctrl:system lib:crypto/bio/bss_file.c:290:
dnsdist_1       | 140157762947896:error:140DC002:SSL routines:use_certificate_chain_file:system lib:ssl/ssl_rsa.c:596:
dnsdist_1       | Error reloading certificates for frontend 0.0.0.0:443: Error setting up TLS context for DoH listener on '0.0.0.0:443': An error occurred while trying to load the TLS server certificate file: /etc/letsencrypt/live/sg-dns1.bancuh.com/fullchain.pem
dnsdist_1       | 140157762947896:error:0200100D:system library:fopen:Permission denied:crypto/bio/bss_file.c:288:fopen('/etc/letsencrypt/live/sg-dns1.bancuh.com/fullchain.pem','r')
dnsdist_1       | 140157762947896:error:20074002:BIO routines:file_ctrl:system lib:crypto/bio/bss_file.c:290:
dnsdist_1       | 140157762947896:error:140DC002:SSL routines:use_certificate_chain_file:system lib:ssl/ssl_rsa.c:596:
dnsdist_1       | Error reloading certificates for frontend [::]:443: Error setting up TLS context for DoH listener on '[::]:443': An error occurred while trying to load the TLS server certificate file: /etc/letsencrypt/live/sg-dns1.bancuh.com/fullchain.pem
dnsdist_1       | 140157762947896:error:0200100D:system library:fopen:Permission denied:crypto/bio/bss_file.c:288:fopen('/etc/letsencrypt/live/sg-dns1.bancuh.com/fullchain.pem','r')
dnsdist_1       | 140157762947896:error:20074002:BIO routines:file_ctrl:system lib:crypto/bio/bss_file.c:290:
dnsdist_1       | 140157762947896:error:140DC002:SSL routines:use_certificate_chain_file:system lib:ssl/ssl_rsa.c:596:
dnsdist_1       | Error reloading certificates for frontend 0.0.0.0:853: An error occurred while trying to load the TLS server certificate file: /etc/letsencrypt/live/sg-dns1.bancuh.com/fullchain.pem
dnsdist_1       | 140157762947896:error:0200100D:system library:fopen:Permission denied:crypto/bio/bss_file.c:288:fopen('/etc/letsencrypt/live/sg-dns1.bancuh.com/fullchain.pem','r')
dnsdist_1       | 140157762947896:error:20074002:BIO routines:file_ctrl:system lib:crypto/bio/bss_file.c:290:
dnsdist_1       | 140157762947896:error:140DC002:SSL routines:use_certificate_chain_file:system lib:ssl/ssl_rsa.c:596:
dnsdist_1       | Error reloading certificates for frontend [::]:853: An error occurred while trying to load the TLS server certificate file: /etc/letsencrypt/live/sg-dns1.bancuh.com/fullchain.pem
dnsdist_1       | reloading ssl cert complete
dnsdist_1       | Closed control connection from 127.0.0.1:57036
ragibkl commented 2 years ago 
root@scw-fr-dns1:~/adblock-dns-server/EXAMPLES/adblock-doh-dot# docker-compose exec dnsdist sh
/ # ls -lah /etc/letsencrypt/live/fr-dns1.bancuh.com/
total 12K    
drwxr-xr-x    2 root     root        4.0K Jun 27 06:55 .
drwx------    3 root     root        4.0K Feb 27 07:44 ..
-rw-r--r--    1 root     root         692 Feb 27 07:44 README
lrwxrwxrwx    1 root     root          42 Jun 27 06:55 cert.pem -> ../../archive/fr-dns1.bancuh.com/cert3.pem
lrwxrwxrwx    1 root     root          43 Jun 27 06:55 chain.pem -> ../../archive/fr-dns1.bancuh.com/chain3.pem
lrwxrwxrwx    1 root     root          47 Jun 27 06:55 fullchain.pem -> ../../archive/fr-dns1.bancuh.com/fullchain3.pem
lrwxrwxrwx    1 root     root          45 Jun 27 06:55 privkey.pem -> ../../archive/fr-dns1.bancuh.com/privkey3.pem
/ # ls -lah /etc/letsencrypt/archive/fr-dns1.bancuh.com/
total 56K    
drwxr-xr-x    2 root     root        4.0K Jun 27 06:55 .
drwx------    3 root     root        4.0K Feb 27 07:44 ..
-rw-r--r--    1 root     root        1.8K Feb 27 07:44 cert1.pem
-rw-r--r--    1 root     root        1.8K Apr 28 06:58 cert2.pem
-rw-r--r--    1 root     root        1.8K Jun 27 06:55 cert3.pem
-rw-r--r--    1 root     root        1.8K Feb 27 07:44 chain1.pem
-rw-r--r--    1 root     root        1.8K Apr 28 06:58 chain2.pem
-rw-r--r--    1 root     root        1.8K Jun 27 06:55 chain3.pem
-rw-r--r--    1 root     root        3.6K Feb 27 07:44 fullchain1.pem
-rw-r--r--    1 root     root        3.6K Apr 28 06:58 fullchain2.pem
-rw-r--r--    1 root     root        3.6K Jun 27 06:55 fullchain3.pem
-rw-------    1 root     root        1.7K Feb 27 07:44 privkey1.pem
-rw-------    1 root     root        1.7K Apr 28 06:58 privkey2.pem
-rw-------    1 root     root        1.7K Jun 27 06:55 privkey3.pem
ragibkl commented 2 years ago

PR: https://github.com/ragibkl/adblock-dns-server/pull/149

It is deployed to all servers. @Tomatoide , please let me know if France servers have any issues at all.

Tomatoide commented 2 years ago

I don't know if this is related, but for a long time now logs doesn't always work especially for secondary servers like fr2 and sg2, shows only ip address at top but query answers section is clear no entries

ragibkl commented 2 years ago

@Tomatoide , I think it might be different problem. I don't use ssl for the logs server. However, the problem we might have is that the doh and dot will stop functioning after 3 months. I restart every 1 or 2 month manually so maybe that's why we did not see the problem yet.

For the logs, I think I set the server up to flush the logs every 10 minutes. That might be the reason you're not seeing logs. Or maybe your device just prioritize server 1 always? Hmm, maybe we should investigate more.

I did a quick test just now on fr2, and can see my logs. image

Let me know if you have any other clues and can reproduce the logs issue on your side again.

ragibkl commented 2 years ago

Somehow, my comments got deleted. The comment I was replying to was deleted as well?

Hello, I can't connect to the Japanese server since yesterday, is it being adjusted?

I don't know why it's not working. I've restarted the server. Should be ok now.

Is it possible to disclose the list of ad-blockers you use? Some websites seem to be blocked, such as mullvad.net, whoer.net, dns.sb (all seem to be related to VPN or DNS?)

You can check the data folder and the configuration.yaml file. I think I remember @Tomatoide added blocklist for some VPNs. That was to prevent kids from easily finding workaround against the porn block.

I can see both my FR2 and SG2 logs, no problem

Thanks for confirming this!

Thank you for providing a free DNS server where you can see your logs!

You're welcome! Credits goes to other contributors as well, especially @Tomatoide for all the help!

ragibkl commented 2 years ago

I still don't know why, but JP1 was having lots of trouble. I might have to bump the server spec or build another one for that region.

BTW @CasanierXI , Are you located in Japan? Is JP1 the closest to you? If so, maybe worth my money to put another server there.

ragibkl commented 2 years ago

@CasanierXI , I figure out what happened to JP1 and why it was not working.

Logs:

Jul 29 03:21:14 li2110-118.members.linode.com kernel: [ 1296]     0  1296      395       13       6        0             0 sleep
Jul 29 03:21:14 li2110-118.members.linode.com kernel: [ 1299]     0  1299   144762   125081     259        0             0 compiler
Jul 29 03:21:14 li2110-118.members.linode.com kernel: Out of memory: Kill process 764 (named) score 620 or sacrifice child
Jul 29 03:21:14 li2110-118.members.linode.com kernel: Killed process 764 (named), UID 100, total-vm:1505820kB, anon-rss:1094916kB, file-rss:0kB, shmem-rss:0kB

Basically, the dns server was taking 1.7 gigs of ram. When the compiler script runs, and it needed to reload the dns, the dns service was killed because it was taking too much memory.

This only started happening recently, when I rolled out the auto self-update feature. Previously, without the self-update, memory usage was fixed.

JP1 only had 2 GiB of ram, and it was not enough. I've bumped the spec on JP1 to 4 GiB. You can use JP1 if it's faster for you.

If you have a bash terminal, please test the following.

ping jp-dns1.bancuh.com
ping sg-dns1.bancuh.com
ping sg-dns2.bancuh.com

In general, you want to use the servers closest to your location by ping. But of course, you can use sg1 & sg2 as well if that's what you like.

ragibkl commented 2 years ago

Update on the ssl cert issue. I did a force renew of the certs on SG1, and forced it to reload it. Seems like no issues. I'm personally using sg1 on my Android phone, so I think we don't have to wait 1 - 2 months. I think the fix works.

I'm closing this issue for now, but if anything breaks, will open a separate ticket to investigate.

ghost commented 2 years ago

JP1 is now available again 👍 Thank you!😄

ragibkl commented 2 years ago

@CasanierXI , ever since I deployed the auto-update feature, jp1 has been unstable. Maybe my update script taking too much memory?

I think we should track this as a new issue.

ragibkl commented 2 years ago

@CasanierXI , I create new ticket here: https://github.com/ragibkl/adblock-dns-server/issues/154 We can keep posting updates there.