Neilpang
commented
6 years ago @sjau do you have any comments ?
Open tschachtner opened 6 years ago
Neilpang
commented
6 years ago @sjau do you have any comments ?
sjau
commented
6 years ago DNS Zone Serial is not handled by the API so nothing to change there... never encountered that issue but I don't use multi-server setup.
Please also post your complete zone file info. I noticed when there's already an acme txt entry it doesn't work.
tschachtner
commented
6 years ago We've seen that the serial number that is used when inserting the TXT record using ISPConfig's remote API is not used for the DNS zone's serial creation. We've already filed a bug report to the ISPConfig team and wait for their response.
sjau
commented
6 years ago Does the master reload the zone with the new added TXT record?
tschachtner
commented
6 years ago Yes, exactly. The master is reloading the zone including the newly created records. But as the zone serial number is not increased, the changes are not replicated to the slave servers. Depending on whether the acme.sh script checks the master or the slave server, the DNS-01 domain check either succeeds or fails. I guess that everything is working fine in a “one DNS server only” scenario.
sjau
commented
6 years ago acme.sh does not have any influence which NS server is being checked by Let's Encrypt. They check one of the NS servers listed in the zone file IIRC.
tschachtner
commented
6 years ago that explains what we currently see. But if this is not fixed and the zone update is not transferred to the slave DNS servers, one cannot reliably use the automated dns registration on ISPConfig systems. I hope the ISPConfig guys are taking care of this quite soon. IMO, DNS update functions do not really make sense if the zone's serial number is not updated.
sjau
commented
6 years ago Well, I also set the limit to 120 seconds before telling LE because ISPC does run it's updater every minute and 1 minute might just not be enough to propagate everything. But for the rest it's up to Timmo Falke and the ISPC Team.
AlphaDE
commented
6 years ago I have the same problem since switching to a master-slave structure of my dns servers. Before, having ispconfig in control of all name servers, the serial did not affect the slaves. But with that above change, the serial needs to be increased so the slave name servers recognize the change in the zone. If I run acme and make a web interface based change to that zone during the wait time, the TXT record gets transmitted and acme finishes sucessful. The api allows to update the zone record including the serial. As the serial should (in case of several changes) only changed once and not everytime, a record change does not automatically trigger a serial change. That's up to the developer of the program using the api.
The zone update function allows to change the serial. The serial is already available, it just needs to get incremented and written back into the zone record again.
I found this elsewhere to properly update the serial but I am more a c/php and not a bash developer.
DATE=$(date +%Y%m%d)
# we're looking line containing this comment
NEEDLE="Serial"
curr=$(/bin/grep -e "${NEEDLE}$" $ZONES_PATH/${ZONE} | /bin/sed -n "s/^\s*\([0-9]*\)\s*;\s*${NEEDLE}\s*/\1/p")
# replace if current date is shorter (possibly using different format)
if [ ${#curr} -lt ${#DATE} ]; then
serial="${DATE}00"
else
prefix=${curr::-2}
if [ "$DATE" -eq "$prefix" ]; then # same day
num=${curr: -2} # last two digits from serial number
num=$((10#$num + 1)) # force decimal representation, increment
serial="${DATE}$(printf '%02d' $num )" # format for 2 digits
else
serial="${DATE}00" # just update date
fi
fi
/bin/sed -i -e "s/^\(\s*\)[0-9]\{0,\}\(\s*;\s*${NEEDLE}\)$/\1${serial}\2/" ${ZONES_PATH}/${ZONE}
In php it would look like
function increase_serial($serial) {
$serial_date = intval(substr($serial, 0, 8));
$count = intval(substr($serial, 8, 2));
$current_date = date("Ymd");
if($serial_date >= $current_date){
$count += 1;
if ($count > 99) {
$serial_date += 1;
$count = 0;
}
$count = str_pad($count, 2, "0", STR_PAD_LEFT);
$new_serial = $serial_date.$count;
} else {
$new_serial = $current_date.'01';
}
return $new_serial;
}and
$soa = $client->dns_zone_get($session_id, ID);
$soa['serial']=increase_serial($soa['serial']);
<updates in the zone>
$client->dns_zone_update($session_id, CLIENT_ID, ID, $soa); the zone get is already performed by the script, so it just needs to increment the zone and write it back, leaving the rest (ns, email, ...) unchanged.
Neilpang
commented
6 years ago @sjau
AlphaDE
commented
6 years ago @Neilpang Any chance to add the update of the serial?
Neilpang
commented
6 years ago @AlphaDE Sorry, I'm not familiar with ispconfig at all. Let's see what @sjau comment.
Thanks.
tschachtner
commented
6 years ago @Neilpang @sjau I'm not an expert on ispconfig or acme.sh, either, but I've proposed a change which might not be the final solution, but maybe help @sjau to update the script...
Please feel free to delete my change request if it does not make sense at all... here's the proposal
Thanks Tom
sjau
commented
6 years ago I only had a brief look at it but looks good so far.
This would only be a temporary suggestion. It seems the the devs at ISPC are also willing to add a zone incremental thing directly through the api:
https://www.howtoforge.com/community/threads/remote-user-api-what-does-dns-zone-functions-do.78079/#post-370541 & https://www.howtoforge.com/community/threads/remote-user-api-what-does-dns-zone-functions-do.78079/#post-370548
AlphaDE
commented
6 years ago @tschachtner One remark...
newZoneSerial=$((curSerialOld + 1))
will mostly work... but you should add a check for the casse that the last two digits are 99. In the case the new serial is composed by increasing to the following day (based on the day presently in the serial) and the counter is 00.
kordianbruck
commented
6 years ago An serial update option was just added to ispconfig 3.1.12: https://git.ispconfig.org/ispconfig/ispconfig3/issues/4806
So a fix for this should be easy now.
Edit: PR is up :tada:
Dear all,
i found some strange phenomenon when using the DNS API functions to automatically do a domain validation for my certificates using an ISPConfig DNS server.
When enrolling for a new certificate, the dns_ispconfig.sh script automatically inserts validation DNS TXT records on my ISPConfig DNS server which is hosting the DNS domain the certificate is meant for. Unfortunately, the following log entries are generated on the ISPConfig server after insertion:
named[xxxxx]: zone example.com/IN: zone serial (2018011622) unchanged. zone may fail to transfer to slaves.After some debugging I found out, that the dns_ispconfig.sh is generating a serial based on the UNIX timestamp value of the current time. While this would be a perfect mechanism to generate a zone serial, ISPConfig is using a different method. They use YYYYMMDDxx (with xx being an automatically increasing rolling number) as serial number. ISPConfig's serial number starts with 2018..... whereas the serial inserted by dns_ispconfig.sh starts with 151.... which is smaller and thus the serial number of the zone is not increased. This means that the zone updates are not transferred to the slaves and the domain validation will fail in some cases (if acme.sh contacts one of the slave name servers).
I can think of two different solutions:
Steps to reproduce
enroll a new certificate using acme.sh and DNS API script dns_ispconfig.sh for a zone that is managed by an ISPConfig DNS server
/root/.acme.sh/acme.sh --issue --force --dns dns_ispconfig -d domain.example.com --test --debugcheck the syslog on the ISPConfig server. You will see above mentioned warning which means that the new zone updates are not sent to the slaves
check the zone file's serial number. You will see that it was not changed by the dns_ispconfig.sh record insertion.
Debug log