github-actions[bot]
commented
1 year ago Closed peterNordin closed 1 year ago
github-actions[bot]
commented
1 year ago 
nemchik
commented
1 year ago Can you try and set the environment variable:
- PROPAGATION=60Or maybe 120.
I'm wondering if the default (which I believe is 30 second) isn't long enough.
peterNordin
commented
1 year ago I tried both 60 and 120, and verified that the env. variable was set in the container with docker exec -it swag printenv but sadly it did not help. I get the same errors in the log.
Regarding the /config/keys/cert.key write problem, I manually created /config/etc/letsencrypt/live/mysubdomain.duckdns.org directory, and then a certifiacte was generated and written. But when I visit my page it complains that the certificate is a self-signed one by Linuxserver.io, so it seems the letsencrypt one was not created in this case.
I have a feeling the two problems are related, but I am not sure.
nemchik
commented
1 year ago Ok, I've tried to reproduce this but I can't.
docker compose up -d, the container started but couldn't get a cert because duckdns.ini didn't contain my token (file had to be created on first run)docker restart swag, the container started and obtained a certificate without issueI did not have to manually create any folders. The init scripts do have some strange looking messages in them that I will investigate, but it worked.
peterNordin
commented
1 year ago OK, thank you for testing, I will keep looking at my setup and see if I can find anything that is odd. One thing I forgot to mention is that swag in this case is not reachable from the outside (public internet). It is used for my internal LAN-only containers. But I guess that should not be a problem since I use DNS validation (and it worked before)
nemchik
commented
1 year ago swag in this case is not reachable from the outside (public internet).
I tested without adding any ports to the docker container as well (pretty much self contained). Didn't have issues. I'd like to figure out what's happening and fix it, but I need to find a way to reproduce it.
adgoncal
commented
1 year ago I have the same issue as OP. I don't know if this makes a difference, but duckdns points to my internal LAN IP for the docker host instead of my WAN IP because I have no intention of exposing anything outside my LAN.
version: "3.9"
networks:
default:
name: swag
driver: bridge
ipam:
config:
- subnet: 172.52.0.0/24
services:
swag:
image: "linuxserver/swag:latest"
container_name: "swag"
cap_add:
- NET_ADMIN
restart: unless-stopped
volumes:
- ${DOCKERDIR}/swag:/config
- ${DOCKERDIR}/swag/fail2ban/fail2ban.sqlite3:/dashboard/fail2ban.sqlite3:ro
- ${DOCKERDIR}/swag/log/nginx:/dashboard/logs:ro
environment:
- PUID=${PUID}
- PGID=${PGID}
- TZ=${TZ}
- URL=mysubdomain.duckdns.org
- VALIDATION=dns
- DNSPLUGIN=duckdns
- SUBDOMAINS=wildcard
# - STAGING=true
- DOCKER_MODS=linuxserver/mods:swag-dashboard|linuxserver/mods:swag-auto-reload
ports:
- 443:443
- 80:80$ docker --version
Docker version 20.10.20, build 9fdeb9c
$ docker-compose --version
docker-compose version 1.28.6, build 5db8d86f
$ uname -a
Linux mynas 4.19.0-21-amd64 #1 SMP Debian 4.19.249-2 (2022-06-30) x86_64 GNU/LinuxEverything works fine on linuxserver/swag:1.31.0-ls153 and the old environment variables, but it broke with linuxserver/swag:1.31.0-ls154.
peterNordin
commented
1 year ago I was finally able to solve the problem.
I turns out that I had set a local override for *.mysubdomain.duckdns.org in my router dns (OPNSens with Unbound) because if I am on my LAN why should I ask duckdns.org about the IP if I already know it. (This is a DNS entry pointing to a private LAN only ip 192.168.1.xxx)
If I disabled that dns override, I am able to get a certificate (I have only tested with a staging certificate, but it will likely work with a proper one to).
peterNordin
commented
1 year ago Now for my guess as to why:
If we look in the letsencrypt logs it tells us there is an exception at line 82 in the duckdns plugin for certbot. This is where the plugin requests the current TXT record before setting the new one for validation. So this happens before the PROPGATION (sleep) takes effect. I guess (I am very unsure) that the python dns package used by the duckdns plugin, ends up getting redirected to my local IP, instead of going to the actual duckdns server to get the TXT record.
adgoncal
commented
1 year ago I was finally able to solve the problem.
I turns out that I had set a local override for *.mysubdomain.dockdns.org in my router dns (OPNSenso with Unbound) because if I am on my LAN why should I ask duckdns.org about the IP if I already know it. (This is dns entry pointing to a private LAN only ip 192.168.1.xxx)
If I disabled that dns override, I am able to get a certificate (I have only tested with a staging certificate, but it will likely work with a proper one to).
That's interesting, I have a similar override in my network. I did not think it would cause any issues because it works fine on 1.31.0-ls153 and earlier.
Why should the DNS challenge require swag to reach that address? I want https in my LAN, but I also want to use my own DNS resolver (bind9 in this case, but it might as well be adguard home or pihole).
peterNordin
commented
1 year ago I have no idea what I am talking about, but it might have something to do with "chaining" whatever that is.
I looked in the code here https://dnspython.readthedocs.io/en/latest/_modules/dns/resolver.html
Search for: self.rrset = self.chaining_result.answer
If rrset is NONE later in the code, the code will throw the exceptions we see in the letsencrypt logs
nemchik
commented
1 year ago @peterNordin excellent find! I don't have any record like this, so I didn't experience the same issue.
This tells me that the issue it's actually something we should open with the plugin to resolve this behavior.
@adgoncal this issue wouldn't have been present in ls153 and below because swag was using a custom implementation for duckdns at that point. ls154 introduced the duckdns plugin in swag.
peterNordin
commented
1 year ago I have been running this now since yesterday and restarted several times. Since I now have a valid certificate the odd problem with keys being re-generated on every restart and failures about writing to disk, no longer happens. I consider the issue solved.
I will consider opening an issue in the duckdns plugin repo.
adgoncal
commented
1 year ago I have been running this now since yesterday and restarted several times. Since I now have a valid certificate the odd problem with keys being re-generated on every restart and failures about writing to disk, no longer happens. I consider the issue solved.
I will consider opening an issue in the duckdns plugin repo.
Let's hope it does not cause any issues when it's time for certificate renewal... 🤞
saxicek
commented
1 year ago I tried both 60 and 120, and verified that the env. variable was set in the container with
docker exec -it swag printenvbut sadly it did not help. I get the same errors in the log.Regarding the
/config/keys/cert.keywrite problem, I manually created/config/etc/letsencrypt/live/mysubdomain.duckdns.orgdirectory, and then a certifiacte was generated and written. But when I visit my page it complains that the certificate is a self-signed one by Linuxserver.io, so it seems the letsencrypt one was not created in this case.I have a feeling the two problems are related, but I am not sure.
I also experienced the error req: Can't open "/config/keys/cert.key" for writing, No such file or directory which was caused by missing folders live/mydomain.com under /etc/letsencrypt. That causes other failures in Certbot, but all went smoothly once I did:
/etc/letsencrypt# mkdir -pv live/mydomain.com@nemchik Do you know in which script these folders should be created?
nemchik
commented
1 year ago @nemchik Do you know in which script these folders should be created?
https://github.com/linuxserver/docker-baseimage-alpine-nginx/blob/7a87255ea6721832754881f3bca4745dda39654b/root/etc/cont-init.d/11-folders#L5 creates /config/keys
https://github.com/linuxserver/docker-swag/blob/d00d2dbe95276f9ea67b4d5d3d6a7b0384538208/root/etc/cont-init.d/40-folders#L6 creates /config/etc/letsencrypt/renewal-hooks which is an adjacent folder to /config/etc/letsencrypt/live (the important thing here is the parent folder exists when it is needed below)
https://github.com/linuxserver/docker-swag/blob/d00d2dbe95276f9ea67b4d5d3d6a7b0384538208/root/etc/cont-init.d/50-certbot#L188-L193 symlinks /etc/letsencrypt/live/domain.com to /config/etc/letsencrypt/live/domain.com (at this point, nothing exists at /etc/letsencrypt/live/domain.com, so the links is a dead link until the next part happens)
https://github.com/linuxserver/docker-swag/blob/d00d2dbe95276f9ea67b4d5d3d6a7b0384538208/root/etc/cont-init.d/50-certbot#L262 creates /etc/letsencrypt/live/mydomain.com/all_the_cert_files
The root cause of the issue explained above was that both users who reported having a problem have (or had) a local DNS record configured in their router (or however they manage local DNS) pointing theirdomain.duckdns.org to a local IP address. The DNS plugin that is now used by SWAG is (unnecessarily) doing a DNS lookup which causes the request for a cert to fail. When it fails, it appears the folders get messed up. The best option from that point is to remove the local DNS record, delete /config/keys and /config/etc and restart the container. SWAG should get a new cert and recreate all the necessary folders at that point.
The actions happening in the plugin start here https://github.com/infinityofspace/certbot_dns_duckdns/blob/5679d3486a1797bd4f18f786b99756384cc3c80b/certbot_dns_duckdns/cert/client.py#L79 I believe there is an option to bypass it, but not via CLI parameters (which is how SWAG currently interacts with certbot and plugins). I'm still exploring options.
saxicek
commented
1 year ago Ok. My scenario is that I own domain with CloudFlare nameservers (no DuckDNS), but I use it for my home network, so I have DNS A record with IP address like 192.168.0.45. Therefore the issue with DNS lookup still applies.
nemchik
commented
1 year ago Interesting. I have not seen that issue yet with cloudflare. Can you try deleting /config/etc and /config/keys and restarting the container?
P.S. you might want to set STAGING=true while testing things to make sure it looks like a valid staging cert can be obtained, and then remove that to get a real working cert, just so you don't get rate limited while experimenting.
saxicek
commented
1 year ago As I wrote above - my issue was fixed when I created the folders manually with mkdir -pv live/mydomain.com. After that everything worked and Let's Encrypt certificate was issued (well, had to use also the PROPAGATION=60). Maybe if I created just the live dir it would also start working, not sure.
GeoCookie
commented
1 year ago I had the same problem after updating swag to version 1.31. On my side, I have a local DNS server that responds to queries from my LAN. This allows me to get local IPs when I'm at home.
I solved my problem by adding a dumb TXT entry on my DNS zone. With this entry, certbot is now able to pass the DNS challenge and renew my certificate.
I really don't understand why, but it works 😊
nemchik
commented
1 year ago Please try running lspipepr/swag:1.32.0-pkg-96e270ce-pr-293, it should work with local DNS records.
You may need to delete your /config/keys and /config/etc folders for swag.
GeoCookie
commented
1 year ago Hi @nemchik
I tried what you asked. This works for me (I can get a certificate) BUT the certificate covers my domain and not my subdomain as requested
Also, I needed to increase the propagation time (30 seconds was not enough).
peterNordin
commented
1 year ago @nemchik I tried the new version and it works for me now with my local DNS override re-enabled in my router. Thanks for the fix. I removed my swag configs and restarted, (setup from scratch) and a cert was now successfully acquired as expected (after adding my duckdns token).
Expected Behavior
After the recent update in the way Duckdns is handled I can no longer get a Letsencrypt certificate using Duckdns. It worked fine before that updated. I feel confident that I have been made the necessary configuration changes (According to the updated README)
Current Behavior
I get the following problems reported in the log (docker logs swag)
docker exec -it swag ls -l /config/keys
But
/config/keys/cert.keypoint to/config/etc/letsencrypt/live/mysubdomain.duckdns.org/privkey.pemwhich has been deleted (the entire live directory was removed by the updated swag container).I also see this odd (maybe) behavior. Every time i restart the container, a new key is generated in the /config/etc/letsencrypt/keys directory.
docker exec -it swag ls -l /config/etc/letsencrypt/keys
Then further down at the end of the log
The DNS response does not contain an answer to the question: mysubdomain.duckdns.org. IN TXTI also found a thread about this: https://community.letsencrypt.org/t/swag-no-answer-from-dns-in-txt-duckdns but with no obvious solution except reverting swag to an older version.
Steps to Reproduce
An alternative approach was to remove (rename) the old host-side swag (/config) directory to let swag generate a new set of files on startup.
/config/keys/cert.keycould be written, but then since I had not added my Duckdns token yet I still could not get a certificate.Environment
OS: Ubuntu 22.04.1 LTS CPU architecture: x86_64 How docker service was installed:
From official Ubuntu repo
Command used to create docker container (run/create/compose/screenshot)
I have also verified that /config/dns-conf/duckdns.ini contains my duckdns token
Docker logs
docker logs swag
docker exec -it swag cat /var/log/letsencrypt/letsencrypt.log