pdns 4.9.1 fails TSIG update with client subnet header

[x] This is not a support question, I have read about opensource and will send support questions to the IRC channel, Github Discussions or the mailing list.
[X] I have read and understood the 'out in the open' support policy

Program: pdns
Issue type: Bug report

Short description

It appears pdns 4.8.4 and 4.9.1 are not able to process TSIG update when request includes client subnet header, providing no error message, but closing connection abruptly. In my current setup, dnsdist 1.9.5 is fronting pdns 4.9.1.

Please note non TSIG request with client subnet header are properly processed.

dnsdist configuration:

setVerbose(true);

setACL({'0.0.0.0/0', '::/0'})

addLocal('0.0.0.0:53', {
  reusePort=true,
  tcpFastOpenQueueSize=512,
})

addLocal('[::]:53', {
  reusePort=true,
  tcpFastOpenQueueSize=512,
})

newServer({
  pool="authoritative",
  address='[::1]:29053',
  useClientSubnet=true,
  healthCheckMode='up'
})

addAction(
  AllRule(),
  PoolAction("authoritative")
)

pdns configuration:

loglevel=7
log-dns-details=yes
log-dns-queries=yes

local-address=[::1]:29053

reuseport=yes
tcp-fast-open=1

launch=gmysql
gmysql-socket=/run/mysqld/mysqld.sock
gmysql-user=xxxx
gmysql-password=xxxx
gmysql-dbname=powerdns
gmysql-dnssec=yes

dnsupdate=yes

allow-dnsupdate-from=xxxx:xxxx:xxxx:xxxx::/64
edns-subnet-processing=yes

UDP use case

When dnsdist uses UDP to connect to pdns, dnsdist is reporting the following error:

Jul 20 12:26:18 pdns.internal dnsdist[96450]: Got query for pants-off.xyz|SOA from [xxxx:xxxx:xxxx:xxxx:d847:7e51:510e:61fa]:33405, relayed to [::1]:29053
Jul 20 12:26:19 pdns.internal dnsdist[96450]: Had a downstream timeout from [::1]:29053 () for query for pants-off.xyz|SOA from [xxxx:xxxx:xxxx:xxxx:d847:7e51:510e:61fa]:33405

Although pdns does not report any log or error, even with loglevel=7.

TCP use case

When dnsdist uses UDP to connect to pdns (or forced with tcpOnly=true), dnsdist is reporting the following error:

Jul 20 12:28:21 pdns.internal dnsdist[96491]: Got query for pants-off.xyz|SOA from [xxxx:xxxx:xxxx:xxxx:d847:7e51:510e:61fa]:9306 (DoUDP, 155 bytes), relayed to [::1]:29053
Jul 20 12:28:21 pdns.internal dnsdist[96491]: Got an exception while handling (reading from backend) TCP query from [xxxx:xxxx:xxxx:xxxx:d847:7e51:510e:61fa]:9306: EOF while reading message

Again here, pdns does not report any log or error, even with loglevel=7.

Environment

Operating system: Gentoo
Software version: dnsdist 1.9.5, pdns 4.9.1.
Software source: Operating system repository

Steps to reproduce

See above for configuration detail

Expected behaviour

I would expect:

pdns to report why the request are being rejected
pdns to process the request

Actual behaviour

nsupdate -y ... times out on the client side.

Other information

Setting edns-subnet-processing=true in pdns configuration does not affect the behaviour.
Setting useClientSubnet=false in dnsdist allow requests to be processed successfully.

I have also made an analyziz comparing network level bytes from the client side and what is being received by pdns after dnsdist, there are not notable changes: DNS transaction ID and added client subnet header:

DNS payload in hex format sent by the client:

0000   03 b0 28 00 00 01 00 00 00 01 00 01 09 70 61 6e
0010   74 73 2d 6f 66 66 03 78 79 7a 00 00 06 00 01 02
0020   78 78 c0 0c 00 01 00 01 00 00 00 0f 00 04 7f 00
0030   00 01 04 61 63 6d 65 00 00 fa 00 ff 00 00 00 00
0040   00 3d 0b 68 6d 61 63 2d 73 68 61 32 35 36 00 00
0050   00 66 9b a3 6a 01 2c 00 20 bb d2 36 32 cf 8d ca
0060   3c 6a c8 26 ea b8 58 5f ab 6f 19 9f 87 9f f9 4d
0070   da 28 92 78 a3 8a 46 c7 04 03 b0 00 00 00 00

DNS payload in hex format received by the pdns:

0000   08 00 28 00 00 01 00 00 00 01 00 02 09 70 61 6e
0010   74 73 2d 6f 66 66 03 78 79 7a 00 00 06 00 01 02
0020   78 78 c0 0c 00 01 00 01 00 00 00 0f 00 04 7f 00
0030   00 01 04 61 63 6d 65 00 00 fa 00 ff 00 00 00 00
0040   00 3d 0b 68 6d 61 63 2d 73 68 61 32 35 36 00 00
0050   00 66 9b a3 6a 01 2c 00 20 bb d2 36 32 cf 8d ca
0060   3c 6a c8 26 ea b8 58 5f ab 6f 19 9f 87 9f f9 4d
0070   da 28 92 78 a3 8a 46 c7 04 03 b0 00 00 00 00 00
0080   00 29 02 00 00 00 00 00 00 0f 00 08 00 0b 00 02
0090   38 00 xx xx xx xx xx xx xx

Diff between the two:

@@ -1,9 +1,11 @@
-0000   03 b0 28 00 00 01 00 00 00 01 00 01 09 70 61 6e
+0000   08 00 28 00 00 01 00 00 00 01 00 02 09 70 61 6e
 0010   74 73 2d 6f 66 66 03 78 79 7a 00 00 06 00 01 02
 0020   78 78 c0 0c 00 01 00 01 00 00 00 0f 00 04 7f 00
 0030   00 01 04 61 63 6d 65 00 00 fa 00 ff 00 00 00 00
 0040   00 3d 0b 68 6d 61 63 2d 73 68 61 32 35 36 00 00
 0050   00 66 9b a3 6a 01 2c 00 20 bb d2 36 32 cf 8d ca
 0060   3c 6a c8 26 ea b8 58 5f ab 6f 19 9f 87 9f f9 4d
-0070   da 28 92 78 a3 8a 46 c7 04 03 b0 00 00 00 00
+0070   da 28 92 78 a3 8a 46 c7 04 03 b0 00 00 00 00 00
+0080   00 29 02 00 00 00 00 00 00 0f 00 08 00 0b 00 02
+0090   38 00 xx xx xx xx xx xx xx

PowerDNS / pdns