UDP status "not working" but seems to be working actually

[ashishkotnala29]

This is an issue which happened on one of my devices (Kernel 3.4.0) after an update to pihole v5.8 (and higher).

  [✓] FTL is listening on port 53
     [✗] UDP (IPv4)
     [✓] TCP (IPv4)
     [✗] UDP (IPv6)
     [✓] TCP (IPv6)

  [✓] Pi-hole blocking is enabled

It is also something which is not particularly unique to my device but I'm a bit lost as to why it happens since a month. Here's a few people having the issue on reddit as well: https://www.reddit.com/r/pihole/comments/rxg75n/udp_not_working_after_581_update/?sort=new

I have another device with an even older kernel v3.0.8 and status is fine there. I'm unsure as to why it may be an issue on newer kernel since the p4a image is same on both devices?

Any idea how can do RCA on this?

Thanks!

[ashishkotnala29]

In v5.8+ of pihole script: https://github.com/pi-hole/pi-hole/blob/2ff10fcd0a5464255d9c004541e6b23d7b42662f/pihole#L316

analyze_ports() {
  local lv4 lv6 port=${1}
  # FTL is listening at least on at least one port when this
  # function is getting called
  # Check individual address family/protocol combinations
  # For a healthy Pi-hole, they should all be up (nothing printed)
  lv4="$(ss --ipv4 --listening --numeric --tcp --udp src :${port})"
  if grep -q "udp " <<< "${lv4}"; then
      echo -e "     ${TICK} UDP (IPv4)"
  else
      echo -e "     ${CROSS} UDP (IPv4)"
  fi
  if grep -q "tcp " <<< "${lv4}"; then
      echo -e "     ${TICK} TCP (IPv4)"
  else
      echo -e "     ${CROSS} TCP (IPv4)"
  fi
  lv6="$(ss --ipv6 --listening --numeric --tcp --udp src :${port})"
  if grep -q "udp " <<< "${lv6}"; then
      echo -e "     ${TICK} UDP (IPv6)"
  else
      echo -e "     ${CROSS} UDP (IPv6)"
  fi
  if grep -q "tcp " <<< "${lv6}"; then
      echo -e "     ${TICK} TCP (IPv6)"
  else
      echo -e "     ${CROSS} TCP (IPv6)"
  fi
  echo ""
}

In v5.7 of pihole script: https://github.com/pi-hole/pi-hole/blob/1b809e4e8ea442cd2a3cc54206429fabd58cce05/pihole#L314

analyze_ports() {
  # FTL is listening at least on at least one port when this
  # function is getting called
  echo -e "  ${TICK} DNS service is listening"
  # Check individual address family/protocol combinations
  # For a healthy Pi-hole, they should all be up (nothing printed)
  if grep -q "IPv4.*UDP" <<< "${1}"; then
      echo -e "     ${TICK} UDP (IPv4)"
  else
      echo -e "     ${CROSS} UDP (IPv4)"
  fi
  if grep -q "IPv4.*TCP" <<< "${1}"; then
      echo -e "     ${TICK} TCP (IPv4)"
  else
      echo -e "     ${CROSS} TCP (IPv4)"
  fi
  if grep -q "IPv6.*UDP" <<< "${1}"; then
      echo -e "     ${TICK} UDP (IPv6)"
  else
      echo -e "     ${CROSS} UDP (IPv6)"
  fi
  if grep -q "IPv6.*TCP" <<< "${1}"; then
      echo -e "     ${TICK} TCP (IPv6)"
  else
      echo -e "     ${CROSS} TCP (IPv6)"
  fi
  echo ""
}

[DesktopECHO]

Hi Ashish,

Try updating the iproute package with one from my CentOS7-ARMv7 repo:

I can't reproduce this issue on with any of my devices... No idea if this will help but worth a try.

[ashishkotnala29]

Thanks for the quick compilation of this. I installed it but same thing. The below command doesn't return any udp port status: ss --ipv4 --listening --numeric --tcp --udp src :53

Netid  State   Recv-Q  Send-Q   Local Address:Port   Peer Address:Port Process
tcp    LISTEN  0       32             0.0.0.0:53          0.0.0.0:*

BUT netstat -an | grep 53 works fine as expected

tcp        0      0 0.0.0.0:53              0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:5335          0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:8953          0.0.0.0:*               LISTEN
tcp6       0      0 :::53                   :::*                    LISTEN
udp        0      0 0.0.0.0:53              0.0.0.0:*
udp        0      0 127.0.0.1:5335          0.0.0.0:*
udp6       0      0 :::53                   :::*

I think it might be due to the custom kernel? I do have the source code for it here: https://github.com/ashishkotnala29/android_kernel_xiaomi_dior/tree/cm-14.1

I'm actually not sure where to look about this...

[DesktopECHO]

I do notice that the output from the ss command (in the case of IPv6) is different from distro-supplied version vs the upgraded version of iproute

[android@localhost ~]$ ss --ipv6 --listening --numeric --tcp --udp src :53
Netid State      Recv-Q Send-Q                                                    Local Address:Port                                                                   Peer Address:Port
udp   UNCONN     0      0                                                                  [::]:53                                                                             [::]:*
tcp   LISTEN     0      32                                                                 [::]:53                                                                             [::]:*

[android@localhost ~]$ sudo rpm -Uvh https://github.com/DesktopECHO/CentOS7-ARMv7/raw/main/iproute-5.9.0-4.el7.armv7hl.rpm
Retrieving https://github.com/DesktopECHO/CentOS7-ARMv7/raw/main/iproute-5.9.0-4.el7.armv7hl.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:iproute-5.9.0-4.el7              ################################# [ 50%]
Cleaning up / removing...
   2:iproute-4.11.0-30.el7            ################################# [100%]

[android@localhost ~]$ ss --ipv6 --listening --numeric --tcp --udp src :53
Netid               State                Recv-Q               Send-Q                             Local Address:Port                              Peer Address:Port               Process
udp                 UNCONN               0                    0                                              *:53                                           *:*
tcp                 LISTEN               0                    32                                             *:53                                           *:*
[android@localhost ~]$

What's the output from your affected device?

[ashishkotnala29]

What's the output from your affected device?

It just doesn't return any udp port status for some reason.

[DesktopECHO]

Nothing at all from both versions of ss?

But netstat works?

We're in bizarro-land with this issue now!

FYI... there may be a workaround. In my Pi-Hole for Windows project I had to create a wrapper for ss because it doesn't work as expected in WSL1. It uses netstat.exe to check the port then reformats the ourput into something Pi-hole can use. You can modify this to work in your situation too.

[DesktopECHO]

Do you get any output from:

ss -6ln

[DesktopECHO]

Oh... also check if the output for ss is the same whether you are a regular user or as root.

[ashishkotnala29]

Here you go. This is from the affected device...

[android@dior ~]$ sudo rpm -qa | grep iproute
iproute-4.11.0-30.el7.armv7hl
[android@dior ~]$ ss --ipv6 --listening --numeric --tcp --udp src :53
Netid  State      Recv-Q Send-Q  Local Address:Port                 Peer Address:Port
tcp    LISTEN     0      32               [::]:53                           [::]:*
[android@dior ~]$ ss -6ln
Netid  State      Recv-Q Send-Q  Local Address:Port                 Peer Address:Port
tcp    LISTEN     0      2                [::]:3389                         [::]:*
tcp    LISTEN     0      5               [::1]:4711                         [::]:*
tcp    LISTEN     0      32               [::]:53                           [::]:*
tcp    LISTEN     0      2               [::1]:3350                         [::]:*
tcp    LISTEN     0      128              [::]:22                           [::]:*
[android@dior ~]$ sudo ss --ipv6 --listening --numeric --tcp --udp src :53
Netid  State      Recv-Q Send-Q  Local Address:Port                 Peer Address:Port
tcp    LISTEN     0      32               [::]:53                           [::]:*
[android@dior ~]$ sudo ss -6ln
Netid  State      Recv-Q Send-Q  Local Address:Port                 Peer Address:Port
tcp    LISTEN     0      2                [::]:3389                         [::]:*
tcp    LISTEN     0      5               [::1]:4711                         [::]:*
tcp    LISTEN     0      32               [::]:53                           [::]:*
tcp    LISTEN     0      2               [::1]:3350                         [::]:*
tcp    LISTEN     0      128              [::]:22                           [::]:*
[android@dior ~]$

Here it is when the package is upgraded...

[android@dior git]$ sudo rpm -qa | grep iproute
iproute-5.9.0-4.el7.armv7hl
[android@dior git]$ ss --ipv6 --listening --numeric --tcp --udp src :53
Netid    State     Recv-Q    Send-Q       Local Address:Port        Peer Address:Port    Process
tcp      LISTEN    0         32                       *:53                     *:*
[android@dior git]$ ss -6ln
RTNETLINK answers: Invalid argument
Netid    State     Recv-Q    Send-Q       Local Address:Port        Peer Address:Port    Process
tcp      LISTEN    0         2                        *:3389                   *:*
tcp      LISTEN    0         5                    [::1]:4711                   *:*
tcp      LISTEN    0         32                       *:53                     *:*
tcp      LISTEN    0         2                    [::1]:3350                   *:*
tcp      LISTEN    0         128                      *:22                     *:*
[android@dior git]$ sudo ss --ipv6 --listening --numeric --tcp --udp src :53
Netid    State     Recv-Q    Send-Q       Local Address:Port        Peer Address:Port    Process
tcp      LISTEN    0         32                       *:53                     *:*
[android@dior git]$ sudo ss -6ln
RTNETLINK answers: Invalid argument
Netid    State     Recv-Q    Send-Q       Local Address:Port        Peer Address:Port    Process
tcp      LISTEN    0         2                        *:3389                   *:*
tcp      LISTEN    0         5                    [::1]:4711                   *:*
tcp      LISTEN    0         32                       *:53                     *:*
tcp      LISTEN    0         2                    [::1]:3350                   *:*
tcp      LISTEN    0         128                      *:22                     *:*
[android@dior git]$

[DesktopECHO]

Ok that looks like what we're expecting. What's the output of these commands with the updated iproute package?

[ashishkotnala29]

Ok that looks like what we're expecting. What's the output of these commands with the updated iproute package?

Updated same post...

[ashishkotnala29]

Here's the same from an unaffected device. Also on the latest package.

[android@xtron ~]$ ss --ipv6 --listening --numeric --tcp --udp src :53
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
Netid    State     Recv-Q    Send-Q       Local Address:Port       Peer Address:Port    Process
udp      UNCONN    0         0                        *:53                    *:*
tcp      LISTEN    0         32                       *:53                    *:*
[android@xtron ~]$ ss -6ln
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
Netid    State     Recv-Q    Send-Q       Local Address:Port       Peer Address:Port    Process
udp      UNCONN    0         0                        *:53                    *:*
tcp      LISTEN    0         5                    [::1]:4711                  *:*
tcp      LISTEN    0         32                       *:53                    *:*
tcp      LISTEN    0         2                    [::1]:3350                  *:*
tcp      LISTEN    0         128                      *:22                    *:*
tcp      LISTEN    0         128                  [::1]:8953                  *:*
tcp      LISTEN    0         2                        *:3389                  *:*
[android@xtron ~]$ sudo ss --ipv6 --listening --numeric --tcp --udp src :53
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
Netid    State     Recv-Q    Send-Q       Local Address:Port       Peer Address:Port    Process
udp      UNCONN    0         0                        *:53                    *:*
tcp      LISTEN    0         32                       *:53                    *:*
[android@xtron ~]$ sudo ss -6ln
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
Netid    State     Recv-Q    Send-Q       Local Address:Port       Peer Address:Port    Process
udp      UNCONN    0         0                        *:53                    *:*
tcp      LISTEN    0         5                    [::1]:4711                  *:*
tcp      LISTEN    0         32                       *:53                    *:*
tcp      LISTEN    0         2                    [::1]:3350                  *:*
tcp      LISTEN    0         128                      *:22                    *:*
tcp      LISTEN    0         128                  [::1]:8953                  *:*
tcp      LISTEN    0         2                        *:3389                  *:*
[android@xtron ~]$ rpm -qa | grep iproute
iproute-5.9.0-4.el7.armv7hl
[android@xtron ~]$

This return udp ports.

Also yes, netstat works on both the devices, same output....

(affected device)

[android@dior git]$ netstat -an | grep :53
tcp        0      0 0.0.0.0:53              0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:5335          0.0.0.0:*               LISTEN
tcp6       0      0 :::53                   :::*                    LISTEN
udp        0      0 0.0.0.0:53              0.0.0.0:*
udp        0      0 127.0.0.1:5335          0.0.0.0:*
udp6       0      0 :::53                   :::*

(unaffected device)

[android@xtron ~]$ netstat -an | grep :53                                                         
tcp        0      0 0.0.0.0:53              0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:5335          0.0.0.0:*               LISTEN
tcp6       0      0 :::53                   :::*                    LISTEN
udp        0      0 0.0.0.0:53              0.0.0.0:*
udp        0      0 127.0.0.1:5335          0.0.0.0:*
udp6       0      0 :::53 

[DesktopECHO]

This is suspect:

RTNETLINK answers: Invalid argument

Have a look here and here ....perhaps it doesn't like your IPv6 settings?

[DesktopECHO]

Unfortunately I have to get back to my Real Job, but keep plugging away at it. Good luck!

Worst case you can hack together a wrapper like I did for WSL1

[ashishkotnala29]

I don't think the RTNETLINK answers: Invalid argument is an issue because that was not a problem on original package and unaffected device also shows the same message but works just fine on old and new package both.

I understand, thanks a lot for your time. This can wait since it seems to be a cosmetic issue...

[DesktopECHO]

What's the output from your affected device?

It just doesn't return any udp port status for some reason.

Where did you see this happen?

[ashishkotnala29]

Where did you see this happen?

[android@dior git]$ ss --ipv6 --listening --numeric --tcp --udp src :53
Netid     State      Recv-Q      Send-Q           Local Address:Port           Peer Address:Port     Process
tcp       LISTEN     0           32                           *:53                        *:*
[android@dior git]$

^^^Affected device

[android@xtron ~]$ ss --ipv6 --listening --numeric --tcp --udp src :53
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
Netid    State     Recv-Q    Send-Q       Local Address:Port       Peer Address:Port    Process
udp      UNCONN    0         0                        *:53                    *:*
tcp      LISTEN    0         32                       *:53                    *:*
[android@xtron ~]$

^^^Unaffected device

[DesktopECHO]

Ok last question... what's the output of ss -tulpen on both the affected and unaffected device?

Possibly related: https://weberblog.net/services-listening-on-ipv6-and-ipv4-or-maybe-not

EDIT: Yeah this seems to be totally cosmetic. If it's a bother then a wrapper for ss could work around this if a proper fix cannot be found.

[ashishkotnala29]

Ok last question... what's the output of ss -tulpen on both the affected and unaffected device?

[android@dior git]$ ss -tulpen
Netid   State    Recv-Q   Send-Q     Local Address:Port     Peer Address:Port   Process
tcp     LISTEN   0        5              127.0.0.1:4711          0.0.0.0:*       uid:995 ino:30524 sk:e44c8000
tcp     LISTEN   0        128              0.0.0.0:80            0.0.0.0:*       ino:28214 sk:e2a48000
tcp     LISTEN   0        32               0.0.0.0:53            0.0.0.0:*       uid:995 ino:30516 sk:e44c8580
tcp     LISTEN   0        128              0.0.0.0:22            0.0.0.0:*       ino:28181 sk:e2a48580
tcp     LISTEN   0        128            127.0.0.1:5335          0.0.0.0:*       ino:24145 sk:e44cf900
tcp     LISTEN   0        128            127.0.0.1:8953          0.0.0.0:*       ino:24146 sk:e44cee00
tcp     LISTEN   0        2                      *:3389                *:*       ino:28357 sk:e43492c0
tcp     LISTEN   0        5                  [::1]:4711                *:*       uid:995 ino:28349 sk:e4348c80
tcp     LISTEN   0        32                     *:53                  *:*       uid:995 ino:30518 sk:e436f6c0
tcp     LISTEN   0        2                  [::1]:3350                *:*       ino:30407 sk:e4368000
tcp     LISTEN   0        128                    *:22                  *:*       ino:28182 sk:e4348000
[android@dior git]$

^^^Affected

[android@xtron ~]$ ss -tulpen
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument
Netid  State   Recv-Q  Send-Q   Local Address:Port    Peer Address:Port  Process
udp    UNCONN  0       0              0.0.0.0:53           0.0.0.0:*
udp    UNCONN  0       0            127.0.0.1:5335         0.0.0.0:*
udp    UNCONN  0       0                    *:53                 *:*
tcp    LISTEN  0       5            127.0.0.1:4711         0.0.0.0:*      ino:1334886 sk:ecdd1e00
tcp    LISTEN  0       5                [::1]:4711               *:*      ino:1334884 sk:e417f900
tcp    LISTEN  0       4            127.0.0.1:5037         0.0.0.0:*      ino:1103 sk:ecdd0000
tcp    LISTEN  0       128            0.0.0.0:80           0.0.0.0:*      ino:1334794 sk:ecdd0500
tcp    LISTEN  0       32                   *:53                 *:*      ino:1333029 sk:e169dd80
tcp    LISTEN  0       32             0.0.0.0:53           0.0.0.0:*      ino:1333027 sk:ecb4b700
tcp    LISTEN  0       2                [::1]:3350               *:*      ino:1334772 sk:e4178b00
tcp    LISTEN  0       128                  *:22                 *:*      ino:1334755 sk:e417e880
tcp    LISTEN  0       128            0.0.0.0:22           0.0.0.0:*      ino:1334754 sk:ecdd2800
tcp    LISTEN  0       128          127.0.0.1:5335         0.0.0.0:*      ino:1332812 sk:ecb48000
tcp    LISTEN  0       128          127.0.0.1:8953         0.0.0.0:*      ino:1332814 sk:ecb4ad00
tcp    LISTEN  0       128              [::1]:8953               *:*      ino:1332813 sk:e169d280
tcp    LISTEN  0       2                    *:3389               *:*      ino:1332885 sk:e169f380
[android@xtron ~]$

^^^Unaffected

As expected, no udp info on affected.

EDIT: Nice find, I had my suspicion of it being a cosmetic issue because pi-hole works as-is and netstat also shows udp ports and the output for it is same on both devices. Only ss fails on one of them. It isn't that big of a deal but IDK if in future on basis of these outputs/checks pihole updates may or may not break.

[DesktopECHO]

Heya I looked at this again real quick. Looks like you need to either patch iproute or have CONFIG_INET_UDP_DIAG enabled in your kernel.

[ashishkotnala29]

Wow, thanks for the config find! Yes, now that I see that config is "not set" in the kernel. I can definitely do that. I'll compile a new kernel with that config enabled and test that.

[ashishkotnala29]

Fixed with the newly compiled kernel with the mentioned config enabled!

CentOS Linux 7 (AltArch) [running via Linux Deploy]

    Pi-hole Stats -> padd
   Pi-hole Update -> pihole -up
  Pi-hole Install -> p4a-install
Pi-hole Uninstall -> p4a-uninstall

  [✓] FTL is listening on port 53
     [✓] UDP (IPv4)
     [✓] TCP (IPv4)
     [✓] UDP (IPv6)
     [✓] TCP (IPv6)

  [✓] Pi-hole blocking is enabled

[android@localhost ~]$ ss --ipv6 --listening --numeric --tcp --udp src :53
Netid State      Recv-Q Send-Q        Local Address:Port                       Peer Address:Port
udp   UNCONN     0      0                      [::]:53                                 [::]:*
tcp   LISTEN     0      32                     [::]:53                                 [::]:*
[android@localhost ~]$

Thanks a lot for your time! Much appreciated.