Impossible to store binary RR of the form IN TYPE65XYZ \# LENGTH BINDATA

[croessner]

Upgrading PowerDNS authoritative server from version 3.3.x to 3.4.x, it seems that has become impossible to store binary data with a custom generic resource record type.

Affects pdns-3.4.2 Some details of the name server:

emerge --info
Portage 2.2.14 (python 2.7.9-final-0, hardened/linux/amd64/no-multilib, gcc-4.8.3, glibc-2.19-r1, 3.17.7-hardened-r1 x86_64)
=================================================================
System uname: Linux-3.17.7-hardened-r1-x86_64-QEMU_Virtual_CPU_version_2.0.0-with-gentoo-2.2
KiB Mem:     1019892 total,     86696 free
KiB Swap:    6290428 total,   6216196 free
Timestamp of tree: Sat, 28 Mar 2015 22:15:01 +0000
ld GNU ld (Gentoo 2.24 p1.4) 2.24
ccache version 3.1.9 [disabled]
app-shells/bash:          4.2_p53
dev-lang/perl:            5.20.1-r4
dev-lang/python:          2.7.9-r1, 3.4.1
dev-util/ccache:          3.1.9-r4
dev-util/cmake:           2.8.12.2-r1
dev-util/pkgconfig:       0.28-r1
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.13.11
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.69
sys-devel/automake:       1.13.4
sys-devel/binutils:       2.24-r3
sys-devel/gcc:            4.8.3
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.4
sys-devel/make:           4.1-r1
sys-kernel/linux-headers: 3.18 (virtual/os-headers)
sys-libs/glibc:           2.19-r1
Repositories: gentoo croessner
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--keep-going --with-bdeps=y --binpkg-respect-use=y --usepkg=y --rebuilt-binaries=y --rebuilt-binaries-timestamp=20140405050000"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://de-mirror.org/gentoo/ rsync://de-mirror.org/gentoo/"
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="acl adns aio amd64 bacula-clientonly bacula-console bash-completion berkdb bindist btrfs bzip2 caps cli cracklib crypt curl cxx device-mapper dri gdbm hardened iconv ipv6 justify libav logrotate loop-aes lzo mmap mmx mmxext modules ncurses nls nptl nscd ntp openmp openssl pam pax_kernel pcre readline session sse sse2 ssl tcpd threads unicode urandom vim-syntax xattr xtpax zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="de en" NGINX_MODULES_HTTP="access auth_basic autoindex browser charset dav empty_gif fastcgi geo gzip headers_more limit_conn limit_req map memcached proxy referer rewrite scgi spdy split_clients ssi upstream_ip_hash userid uwsgi" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby20 ruby19" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON

Running MariaDB as a SQL backend to PDNS:

Version 10.0.16

If you try to store binary data like the following example and rectify the zone, a pdnssec check-zone will raise a warning about problems with the stored value and the content:

mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 17597
Server version: 10.0.16-MariaDB-log Source distribution

Copyright (c) 2000, 2014, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> use pdns;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
MariaDB [pdns]> insert into records (domain_id, name, type, content, ttl) values (21603, "1053463d383dc1e87f06fff34b3c6a2d340d91e184d46d70144ffa5a._encr._smimecert.roessner.co", "TYPE65514", "\\# 1303 010000003082050F308203F7A003020102021069CCB1DF3BD4256F7F869BBBB9100BC4300D06092A864886F70D01010B0500308197310B3009060355040613024742311B30190603550408131247726561746572204D616E636865737465723110300E0603550407130753616C666F7264311A3018060355040A1311434F4D4F444F204341204C696D69746564313D303B06035504031334434F4D4F444F2052534120436C69656E742041757468656E7469636174696F6E20616E642053656375726520456D61696C204341301E170D3134313231303030303030305A170D3135313231303233353935395A301E311C301A06092A864886F70D010901160D6340726F6573736E65722E636F30820122300D06092A864886F70D01010105000382010F003082010A0282010100BAE3DAF1760CE1073CCC742FA4E91422CCFD4C3CCB40EE98FFDE1F012E7357E8EEF4D119056F5ACEA799C8F4D4211AC1A27A580CE8C092F88A6456A48DB234DFC31C5C07B760BE20110006ADE9FD173AACA244DCC616468286C963BBF7CB7A28A073DBA05AB71D9711ECFA36C2696C2B8BC248608DF580161C82B579E5513918245AA17F3D6798E75828E8C1EBF0E0482B048925E451FD515524BD60234519ED3F341451FA206AB75B55B3386951E81B8021912E3439B8CA33BBCE34003DE79C3CB47F8276B682849F1AB9EB61A7BCAECC49A1000C58672C42CAD2622C4503185B56ACEF2E61777C326C918D3284D15C73C893AB0A2ED93A2BE6CD578FCC4A530203010001A38201CD308201C9301F0603551D2304183016801482AF6C8CF8C5FE96617CE81F3D2B71485EC48BC0301D0603551D0E04160414AA5D6DE47CD6A999E2EDC4FBA22AE542A437528D300E0603551D0F0101FF0404030205A0300C0603551D130101FF04023000301D0603551D250416301406082B0601050507030406082B0601050507030230460603551D20043F303D303B060C2B06010401B2310102010305302B302906082B06010505070201161D68747470733A2F2F7365637572652E636F6D6F646F2E6E65742F435053305A0603551D1F04533051304FA04DA04B8649687474703A2F2F63726C2E636F6D6F646F63612E636F6D2F434F4D4F444F525341436C69656E7441757468656E7469636174696F6E616E64536563757265456D61696C43412E63726C30818B06082B06010505070101047F307D305506082B060105050730028649687474703A2F2F6372742E636F6D6F646F63612E636F6D2F434F4D4F444F525341436C69656E7441757468656E7469636174696F6E616E64536563757265456D61696C43412E637274302406082B060105050730018618687474703A2F2F6F6373702E636F6D6F646F63612E636F6D30180603551D110411300F810D6340726F6573736E65722E636F300D06092A864886F70D01010B0500038201010055E6C263771F1893AFFEF0BC3C20269402ED300DA5C6961507AA1A04565E2187D814D0ED3422A9B1E2F3B85AFFB37BB3671805093A612AD268343D3F9B585A3E54F9428B2B4670AE991A04A68BAB30C8EB230E72A39B92702E6A16C564BF46E88827CEB1C202AE408D75A9987696B8144D3ACA1E5C44AE0CC8888DFB7D1CDCFF7F494848801C1095B8D6C39B6ADA143C39E600EE2805A59FFB7027362A764405FDEEDEB25026F7DE79A2D1D68C8CC60DA8ECCA3F3549D304774E0AC6C5D20BC1D245D79BA590FE7F2BC4918447C88766B3AE09726EE317C4F2CA639BCD4F13992DEEA2009B0405B575E234B78523387B7CF011F55D335E750286D83E30C2A0DB", 3600);
Query OK, 1 row affected (0.01 sec)

pdnssec rectify-zone roessner.co
Adding NSEC ordering information

Checking the zone fails:

pdnssec check-zone roessner.co
[Warning] Parsed and original record content are not equal: 1053463d383dc1e87f06fff34b3c6a2d340d91e184d46d70144ffa5a._encr._smimecert.roessner.co IN TYPE65514 '\# 1303 010000003082050F... A0DB' (Content parsed as '\# 2614 5c2... 04442')
Checked 12 records of 'roessner.co', 0 errors, 1 warnings.

I have cut the long lines just to reflect the main output.

Testing a TXT RR, it works. So it must have to do with the binary data.

My records table looks like this, after having done the ALTER TABLE stuff from the dense-script:

CREATE TABLE IF NOT EXISTS `records` (
  `id` int(11) NOT NULL,
  `domain_id` int(11) DEFAULT NULL,
  `name` varchar(255) DEFAULT NULL,
  `type` varchar(10) DEFAULT NULL,
  `content` varchar(64000) DEFAULT NULL,
  `ttl` int(11) DEFAULT NULL,
  `prio` int(11) DEFAULT NULL,
  `change_date` int(11) DEFAULT NULL,
  `disabled` tinyint(1) DEFAULT '0',
  `ordername` varchar(255) CHARACTER SET latin1 COLLATE latin1_bin =
DEFAULT NULL,
  `auth` tinyint(1) DEFAULT '1'
) ENGINE=3DInnoDB AUTO_INCREMENT=3D17435 DEFAULT CHARSET=3Dlatin1;

Anything else works.

You can find more details on the mailing list pdns-users.

[cmouse]

did you try without space after #?

[croessner]

pdnssec check-zone roessner.co

[Error] Following record had a problem: 1053463d383dc1e87f06fff34b3c6a2d340d91e184d46d70144ffa5a._encr._smimecert.roessner.co IN TYPE65514 \#1303 010000003... A0DB
[Error] Error was: Unknown record was stored incorrectly, need 3 fields, got 2: \#1303 010000003...

It requires 3 fields

[croessner]

The backslash here on Github was not displayed. Sorry It is #1303

[cmouse]

did you notice your data is 2606 long. Afaik the length is storage length.

[croessner]

Am 29.03.2015 um 14:58 schrieb Aki Tuomi notifications@github.com:

did you notice your data is 2606 long. Afaik the length is storage length.

— Reply to this email directly or view it on GitHub.

This brings this error:

[Error] Following record had a problem: 1053463d383dc1e87f06fff34b3c6a2d340d91e184d46d70144ffa5a._encr._smimecert.roessner.co IN TYPE65514 \# 2606 010000003082050F308203F7A003020102021069CCB1DF3BD4256F7F869BBBB9100BC4300D06092A864886F70D01010B0500308197310B3009060355040613024742311B30190603550408131247726561746572204D616E636865737465723110300E0603550407130753616C666F7264311A3018060355040A1311434F4D4F444F204341204C696D69746564313D303B06035504031334434F4D4F444F2052534120436C69656E742041757468656E7469636174696F6E20616E642053656375726520456D61696C204341301E170D3134313231303030303030305A170D3135313231303233353935395A301E311C301A06092A864886F70D010901160D6340726F6573736E65722E636F30820122300D06092A864886F70D01010105000382010F003082010A0282010100BAE3DAF1760CE1073CCC742FA4E91422CCFD4C3CCB40EE98FFDE1F012E7357E8EEF4D119056F5ACEA799C8F4D4211AC1A27A580CE8C092F88A6456A48DB234DFC31C5C07B760BE20110006ADE9FD173AACA244DCC616468286C963BBF7CB7A28A073DBA05AB71D9711ECFA36C2696C2B8BC248608DF580161C82B579E5513918245AA17F3D6798E75828E8C1EBF0E0482B048925E451FD515524BD60234519ED3F341451FA206AB75B55B3386951E81B8021912E3439B8CA33BBCE34003DE79C3CB47F8276B682849F1AB9EB61A7BCAECC49A1000C58672C42CAD2622C4503185B56ACEF2E61777C326C918D3284D15C73C893AB0A2ED93A2BE6CD578FCC4A530203010001A38201CD308201C9301F0603551D2304183016801482AF6C8CF8C5FE96617CE81F3D2B71485EC48BC0301D0603551D0E04160414AA5D6DE47CD6A999E2EDC4FBA22AE542A437528D300E0603551D0F0101FF0404030205A0300C0603551D130101FF04023000301D0603551D250416301406082B0601050507030406082B0601050507030230460603551D20043F303D303B060C2B06010401B2310102010305302B302906082B06010505070201161D68747470733A2F2F7365637572652E636F6D6F646F2E6E65742F435053305A0603551D1F04533051304FA04DA04B8649687474703A2F2F63726C2E636F6D6F646F63612E636F6D2F434F4D4F444F525341436C69656E7441757468656E7469636174696F6E616E64536563757265456D61696C43412E63726C30818B06082B06010505070101047F307D305506082B060105050730028649687474703A2F2F6372742E636F6D6F646F63612E636F6D2F434F4D4F444F525341436C69656E7441757468656E7469636174696F6E616E64536563757265456D61696C43412E637274302406082B060105050730018618687474703A2F2F6F6373702E636F6D6F646F63612E636F6D30180603551D110411300F810D6340726F6573736E65722E636F300D06092A864886F70D01010B0500038201010055E6C263771F1893AFFEF0BC3C20269402ED300DA5C6961507AA1A04565E2187D814D0ED3422A9B1E2F3B85AFFB37BB3671805093A612AD268343D3F9B585A3E54F9428B2B4670AE991A04A68BAB30C8EB230E72A39B92702E6A16C564BF46E88827CEB1C202AE408D75A9987696B8144D3ACA1E5C44AE0CC8888DFB7D1CDCFF7F494848801C1095B8D6C39B6ADA143C39E600EE2805A59FFB7027362A764405FDEEDEB25026F7DE79A2D1D68C8CC60DA8ECCA3F3549D304774E0AC6C5D20BC1D245D79BA590FE7F2BC4918447C88766B3AE09726EE317C4F2CA639BCD4F13992DEEA2009B0405B575E234B78523387B7CF011F55D335E750286D83E30C2A0DB
[Error] Error was: invalid unknown record length for label : size not equal to length field (2606 != 5212)
Checked 12 records of 'roessner.co', 1 errors, 0 warnings.

And notice that the binary data I store is already HEX encoded, so two nibbles form one byte.

[croessner]

Some background:

The IETF currently works on a draft for SMIMEA. And I need to do tests. So what is stored is the following:

The first 4 octets are:

• cert_usage_field
• selector
• matching_type
• cert_access_field

So in my case 1 0 0 0, which is 01000000 The rest is a DER x509 certificate encoded in HEX. This is what starts with 308205

As the IANA has not already finished the new SMIMEA RR (or maybe they did in the last days?), "we" work with a temporary generic type.

This works perfectly with the BIND name server. And this worked in PDNS 3.3.x

[croessner]

Super simple example, which you can test:

MariaDB [pdns]> insert into records (domain_id, name, type, content, ttl) values (21603, "test._sub2._sub1.roessner.co", "TYPE65226", "\\# 3 414243", 3600);
Query OK, 1 row affected (0.02 sec)

MariaDB [pdns]> Bye
deltaweb root@ns1  ~ # pdnssec rectify-zone roessner.co
Adding NSEC ordering information
deltaweb root@ns1  ~ # pdnssec check-zone roessner.co
[Warning] Parsed and original record content are not equal: test._sub2._sub1.roessner.co IN TYPE65226 '\# 3 414243' (Content parsed as '\# 11 5c23203320343134323433')
Checked 12 records of 'roessner.co', 0 errors, 1 warnings.

414243 is Decimal: 65 66 67 is ABC

I chose TYPE65226, just to show that RR does not matter

[cmouse]

It seems that we encode the data again when doing pdns/pdnssec check-all-zones, but would appear to work right when asked with dig.

[croessner]

Now I see what PDNS does:

Look at the sequence: 5c23203320343134323433 5c - backslash 23 - # 20 - SPACE 33 - 3 20 - SPACE ...

Yes. You are right ;-)

[croessner]

Ok, so storing works, but the check-test is broken. So it simply is a bug in the pdnssec part and not in pdns itself

[cmouse]

@croessner can you test now?