Odd behaviour when RRSIG fetched from upstream DNS

[Taomyn]

I'm using DnsServer 7.1 as my local DNS but for some reason since doing this several applications seem to have strange issues, most notably in my case my Brave browser and Remote Desktop Manager (RDM) my RDP application.

With Brave when I visit a site I have not opened in a while or after re-opening the browser, I will get an error that the DNS could not be resolved shortly getting auto-refreshed and the page appearing as normal - unless it was trying to switch connections from HTTP to HTTPS in which case it pauses saying it was unable to switch and should I proceed. It's not an easy thing to capture, however with RDM I've been able to capture some information - it complains it cannot get to my Bitwarden account for credentials when I open a new remote session but if I retry immediately it works.

In the DnsServer logs I have this:

[2021-11-30 15:02:37 Local] [[::1]:0] [admin] Cached zone was deleted: vault.bitwarden.com
[2021-11-30 15:02:41 Local] [[::1]:0] [admin] Cached zone was deleted: bitwarden.com
[2021-11-30 15:02:54 Local] [127.0.0.1:61363] [UDP] QNAME: vault.bitwarden.com; QTYPE: A; QCLASS: IN; RCODE: NoError; ANSWER: [104.18.13.33, 104.18.12.33, QSAxMyAzIDMwMCAyMDIxMTIwMTE1MDAzOCAyMDIxMTEyOTEzMDAzOCAzNDUwNSBiaXR3YXJkZW4uY29tLiBJYTlzUDFhakc1SGc4eStVOERidHBnL2VieEUxUmlyY3dpVzAyVXlOQzh1VkJwYlB6ZU5qcGM5MzNuSnFZS1FXUUI5N2ZrazNMenMvLys4RUhyV0V1UT09]
[2021-11-30 15:02:54 Local] [127.0.0.1:49759] [UDP] QNAME: vault.bitwarden.com; QTYPE: A; QCLASS: IN; RCODE: NoError; ANSWER: [104.18.13.33, 104.18.12.33, QSAxMyAzIDMwMCAyMDIxMTIwMTE1MDAzOCAyMDIxMTEyOTEzMDAzOCAzNDUwNSBiaXR3YXJkZW4uY29tLiBJYTlzUDFhakc1SGc4eStVOERidHBnL2VieEUxUmlyY3dpVzAyVXlOQzh1VkJwYlB6ZU5qcGM5MzNuSnFZS1FXUUI5N2ZrazNMenMvLys4RUhyV0V1UT09]
[2021-11-30 15:03:04 Local] [127.0.0.1:49759] [UDP] QNAME: vault.bitwarden.com; QTYPE: A; QCLASS: IN; RCODE: NoError; ANSWER: [104.18.13.33, 104.18.12.33]
[2021-11-30 15:03:04 Local] [127.0.0.1:61363] [UDP] QNAME: vault.bitwarden.com; QTYPE: A; QCLASS: IN; RCODE: NoError; ANSWER: [104.18.13.33, 104.18.12.33]

And from the NirSoft tool, DNS Lookup View I get the following entries for the Bitwarden lookups:

Host Name   Query Type  Query Status    Query Result    Other Query Result  Process ID  Thread ID   Process Name    Process Path    Time    
vault.bitwarden.com A   Error 87            7008    14876   RemoteDesktopManager64.exe  C:\Program Files (x86)\Devolutions\Remote Desktop Manager\RemoteDesktopManager64.exe    30/11/2021 15:02:54 
vault.bitwarden.com AAAA    Error 1460          7008    14876   RemoteDesktopManager64.exe  C:\Program Files (x86)\Devolutions\Remote Desktop Manager\RemoteDesktopManager64.exe    30/11/2021 15:02:54 
vault.bitwarden.com A   Error 87            7008    5076    RemoteDesktopManager64.exe  C:\Program Files (x86)\Devolutions\Remote Desktop Manager\RemoteDesktopManager64.exe    30/11/2021 15:03:04 
vault.bitwarden.com AAAA    Succeeded   ::ffff:104.18.13.33, ::ffff:104.18.12.33        7008    5076    RemoteDesktopManager64.exe  C:\Program Files (x86)\Devolutions\Remote Desktop Manager\RemoteDesktopManager64.exe    30/11/2021 15:03:04 

Any ideas? All the time DnsServer is logging entries without the RRSIG there is no issue. it's only the first time after I clear the DnsServer cache where it includes the RRSIG that it happens.

If I revert my DNS server settings of my LAN card back to the main network DNS all is fine and none of the above occurs, it's only when I use DnsServer that I see problems. FYI the upstream server is my DoH server which you have been helping me with in another issue.

[ShreyasZare]

Thanks for the post. The current v7.1 of the DNS Server is not security-aware so it does not uses DNSSEC nor does it understand DNSSEC records.

The first time that the domain is resolved, the response that is received from the upstream is directly returned to the waiting clients. For the second time, the DNS server will query its cache and only return the records found in the cache for the record type that was requested in the query.

So, it seems that your upstream DoH server is adding RRSIG record to the response when it was not requested by the DNS server since it does not implemented DNSSEC. The response with RRSIG then gets delivered to the clients where the client apps are unable to filter out the RRSIG record from response and failing trying to parse RRSIG as A/AAAA record.

Try querying the upstream DoH server directly using the DNS Client tab and see if the response contains RRSIG record to confirm this.

The upcoming major release of DNS Server will support DNSSEC so it wont be an issue later. It will take more time for the next release though since DNSSEC implementation is taking a lot of time and will require a lot of testing.

[Taomyn]

Yes, a direct query using the DNS client to my DoH server returns RRSIG, for example:

{
  "Metadata": {
    "NameServer": "https://doh.mydomain.com/dns-query (nnn.nnn.nnn.nnn)",
    "Protocol": "HttpsJson",
    "DatagramSize": "778 bytes",
    "RoundTripTime": "109.72 ms"
  },
  "Identifier": 5282,
  "IsResponse": true,
  "OPCODE": "StandardQuery",
  "AuthoritativeAnswer": false,
  "Truncation": false,
  "RecursionDesired": true,
  "RecursionAvailable": true,
  "Z": 0,
  "AuthenticData": false,
  "CheckingDisabled": false,
  "RCODE": "NoError",
  "QDCOUNT": 1,
  "ANCOUNT": 4,
  "NSCOUNT": 0,
  "ARCOUNT": 0,
  "Question": [
    {
      "Name": "www.mitel.com",
      "Type": "A",
      "Class": "IN"
    }
  ],
  "Answer": [
    {
      "Name": "www.mitel.com",
      "Type": "CNAME",
      "Class": "IN",
      "TTL": "1038 (17 mins 18 sec)",
      "RDLENGTH": "33 bytes",
      "RDATA": {
        "Domain": "www.mitel.com.cdn.cloudflare.net"
      }
    },
    {
      "Name": "www.mitel.com.cdn.cloudflare.net",
      "Type": "A",
      "Class": "IN",
      "TTL": "36 (36 sec)",
      "RDLENGTH": "13 bytes",
      "RDATA": {
        "IPAddress": "104.18.20.112"
      }
    },
    {
      "Name": "www.mitel.com.cdn.cloudflare.net",
      "Type": "A",
      "Class": "IN",
      "TTL": "36 (36 sec)",
      "RDLENGTH": "13 bytes",
      "RDATA": {
        "IPAddress": "104.18.21.112"
      }
    },
    {
      "Name": "www.mitel.com.cdn.cloudflare.net",
      "Type": "RRSIG",
      "Class": "IN",
      "TTL": "36 (36 sec)",
      "RDLENGTH": "151 bytes",
      "RDATA": {
        "DATA": "QSAxMyA2IDMwMCAyMDIxMTIwMzEwMTQxMCAyMDIxMTIwMTA4MTQxMCAzNDUwNSBjbG91ZGZsYXJlLm5ldC4gWTg2U2tUTE95ZHRzZzZ0emtjYWwxd2tyZXNrYjZRbVYzYWV4QmVzMHFvcGZzOGFHYVJ1bWJoMitjZ0tnd0g2NitjRi94cXZIU2lKL0VTY2ZJblpPNXc9PQ=="
      }
    }
  ],
  "Authority": [],
  "Additional": []
}

I can't seem to find any way to stop my DoH server providing the RRSIG - it's a Docker/PodMan container running on a Fedora server.

[ShreyasZare]

Thanks for confirming. The DoH upstream server you have seems to have DO flag set by default when reading any request. The current version of DNS Server on the other hand does not even support EDNS so getting back RRSIG record in response is unexpected.

I would suggest that you report this issue to the DoH project so that they can fix this. Meanwhile, since this is your own deployment, you can use another DoH server on the upstream that works correctly.

From my side, the next release is planned to roll out DNSSEC validation support so this issue would get correctly handled.

[Taomyn]

Thanks, I did some digging and further testing using curl. and it seems that my DoH server always returns the RRSIG no matter if I set "do=0" in my query, so I have put an issue ticket on their Github and will see what they say.

Out of curiosity how come your server is not sanitising the data it then provides to the querying client? The data is not even an IP address and totally invalid, yet it seems your caching function is doing exactly that and not storing it.

[ShreyasZare]

Out of curiosity how come your server is not sanitising the data it then provides to the querying client? The data is not even an IP address and totally invalid, yet it seems your caching function is doing exactly that and not storing it.

There is sanitation done but it does not check record types since the response could contain different type of records like CNAME/DNAME etc. The RRSIG record is read as an unknown record, so the DNS server knows its not an A/AAAA record but still reads it and passes it on.

The DNS server cache however works differently as it returns results for the query from its database and since its not DNSSEC aware, it wont check for any RRSIG to include in response. The cache however does store RRSIG record just like any other record which you can check from the Cache tab.

[Taomyn]

Seems not using JSON but IETF fixes the issue, although the way JSON works is probably a bug. Not sure why I was using HTTPS-JSON for my setup.

https://github.com/m13253/dns-over-https/issues/118#issuecomment-985310018

[ShreyasZare]

Seems not using JSON but IETF fixes the issue, although the way JSON works is probably a bug. Not sure why I was using HTTPS-JSON for my setup.

m13253/dns-over-https#118 (comment)

Using the IETF standard version is best way forward since the JSON version is non standard and does not define a lot of DNS record formats.

[ShreyasZare]

Technitium DNS Server v8 is now released which supports DNSSEC and thus fixes this issue. Do check and let me know your feedback.

[Taomyn]

Confirmed working, no longer seeing the errors nor getting browser stalls - thanks