UDP Broadcast Failure and Solution

[formix]

I'm working on a project where I listen for a brodcasted UDP packet on one network, serialize the IP PDU packet and send it to peers on other networks. These peers then rehydrate the IP packet, change the destination address to the broadcasting address of the configured network interface of the second VLAN and use the PacketSender class to broadcast the packet for client machines to setup themselves to reach the server on the other network.

Basically, a legacy application used to have the server and its clients on the same virtual network (on different physical sites). That had to change lately for security/regulation reasons and now clients and servers are on separate VLANs.

On my "broadcasting" machine (linux CentOS), I can clearly see the packet I forged being put on the interface. Since the src IP address is not reachable from the interface I'm echoing the broadcast, I have set the reverse path filtering to 0 for all my interfaces, including for default and all values:

sysctl -w 'net.ipv4.ip_forward=1'
sysctl -w "net.ipv4.conf.all.rp_filter=0"
sysctl -w 'net.ipv4.conf.default.rp_filter=0'
sysctl -w 'net.ipv4.conf.ens33.rp_filter=0'
sysctl -w 'net.ipv4.conf.ens37.rp_filter=0'

Note 1: I know ip_forward should not be involved in that but I set it anyway just in case. Note 2: These settings are changed until the next reboot. I'll find a way to make that permanent once I have a broadcast that works.

Using WireShark, I enabled the checksum validation for IP, and UDP and everything is valid.

I inspected the code in packet_sender.cpp and I don't see anywhere the SO_BROADCAST options being set on the socket, looking at the PacketSender::open_l3_socket method in particular. I'm planning to do a pull request with a patch to enable broadcasting. I'm looking for direction here and constructive comments are welcome as well. Here are my options. If they are not aligned with your idea, please put me back on track ;).

1. Add a getter/setter method `PacketSender::udp_broadcast(void/bool)'?
2. Change PacketSender::open_l3_socket in such a way that if type == SocketType.IP_UDP_SOCKET then I enable broadcast on the socket.
3. Mix (1) and (2)
4. Always enable UDP broadcast whenever a socket is created, no matter the case?
5. I'm totally off track and I missed something that already exist in Tins that will allow me to do what I want.
6. Any other idea?

I will pick case (4) and test it in my environment to validate that it fixes my issue. I'll get back with my results soon.

[formix]

For my simple test case, I have two physical machines:

Machine 01, CentOS in VMWare on a Windows host with direct access to network: Forges an UDP brodcast with an unreacheable source IP. Machine 02, Ubuntu 14, physical separate machine on the same network over wifi.

Using Wireshark on the windows host to witness my forged broadcasts on the physical device (I see my forged packets on my network card). Using tcpdump on the Machine 02 to see my forged packet arriving at me (not seen any yet).

1. Windows firewall is down
2. On machines 01 and 02; iptables and firewald are either down or not installed.

Checked machines and ports availability using nmap all around and everything is fine.

[formix]

Ok, I tested with the simple setup stated in my previous comment. To simplify the case, I set the FROM address to the actual real address of the interface I'm broadcasting from. This is how I set the SO_BROADCAST option in libtins' open_l3_socket method:

Original packet_sender.cpp:

        if (setsockopt(sockfd, level, IP_HDRINCL, (option_ptr)&on, sizeof(on)) != 0) {
            throw socket_open_error(make_error_string());
        }

Modified packet_sender.cpp:

        if (setsockopt(sockfd, level, IP_HDRINCL | SO_BROADCAST, 
                (option_ptr)&on, sizeof(on)) != 0) {
            throw socket_open_error(make_error_string());
        }

Rebuilt, installed and rebooted the machine to make sure everything is reloaded properly. Then I made this little broadcasting program in my linux virutal machine (CentOS 7, VMWare inside a Win10 host):

#include <tins/tins.h>
#include <iostream>
#include <unistd.h>
#include <stdbool.h>

using namespace std;
using namespace Tins;

bool running = true;

void signal_callback(int signum) {
    running = false;
}

int main(int argc, char** argv) {
    cout << "Starting broadcaster...\n";
    signal(SIGINT, signal_callback);
    EthernetII eth = 
        EthernetII() / 
        IP("192.168.1.255", "192.168.1.120") / 
        UDP(55554, 45856) / 
        RawPDU("HELLO!");
    PacketSender sender;
    while(running) {
        cout << "HELLO!\n";
        sender.send(eth);
        sleep(2);
    }
    cout << "\nBroadcasting done, bye!\n";
    return 0;
}

Compilation command line:

c++ crafter.cpp -std=c++11 -ltins -o crafter

The second machine (physical Ununtu 16.04) performs a tcpdump with that command:

tcpdump -s0 -v -nn udp dst port 55554

My Win10 Host machine has the following IP address: 192.168.1.118/24. As you can see, the gest CentOS machine has 192.168.1.120/24.

On the windows host, WireShark listening on the 192.168.1.118 interface sees all broadcasts I'm doing. Somewhere else on the physical network, the Ubuntu machine does not see anything.

I made sure that no firewall is up on any virtual/physical machines twice.

In the end, This is not working. My two guesses are:

1. I didn't do the right thing in libtins source code. I'm either not enabling broadcasting correctly in the open_l3_socket or I do not do it at the right place.
2. Picking blindly a source port for sending is not actually going to work. I may have to ask the OS and do something differently but I don't know what.
3. ((edit)) I'm dumb and I'm not crafting the PDU correctly.

The answer is number 3!((end-edit))

Either case, I'm stuck here and don't know what to do next. Do you have any clue why my broadcasting do not work? Can you help me find a solution?

Thanks!

[formix]

Ok, I'm building from the master branch. That can be an issue. I'll try with the latest release source code tonight.

[formix]

I found my issue!

I built a "standard" broadcast program that would send a packet identical to the packet I'm crafting using libtins. The broadcast worked as expected. By comparing wireshard packet, the difference exploded in my face: the destination hardware address of the working packet was "ff:ff:ff:ff:ff:ff"!!!

The simple fix is to add a simple line to the test program. I will rollback my changes to the socket options to see if it still works afterward.

How to craft an UDP Broadcast

#include <tins/tins.h>
#include <iostream>
#include <unistd.h>
#include <stdbool.h>

using namespace std;
using namespace Tins;

bool running = true;

void signal_callback(int signum) {
        running = false;
}

int main(int argc, char** argv) {
        cout << "Starting broadcaster...\n";
        signal(SIGINT, signal_callback);
        EthernetII eth =
                EthernetII() /
                IP("192.168.1.255", "10.0.25.6") /
                UDP(55554, 45856) /
                RawPDU("HELLO!");
        eth.dst_addr("ff:ff:ff:ff:ff:ff");  // Hardware b-cast address!!!
        NetworkInterface iface("ens33");
        PacketSender sender;
        while(running) {
                cout << "HELLO!\n";
                sender.send(eth, iface);
                sleep(2);
        }
        cout << "\nBroadcasting done, bye!\n";
        return 0;
}

To make sure the kernel will not reject the PDU because the source IP address is not reacheable, you have to do the following for all your interfaces:

sysctl -w "net.ipv4.conf.all.rp_filter=0"
sysctl -w 'net.ipv4.conf.default.rp_filter=0'
sysctl -w 'net.ipv4.conf.eth0.rp_filter=0'
sysctl -w 'net.ipv4.conf.eth1.rp_filter=0'

If the source address is routable by another interface that exists on the machine, you can set rp_filter values to '2' instead.

[formix]

Removed my fix from packet_sender.cpp and everything works as is.

I learned something today :). It was so absolutely obvious! I literally fell on my back when I saw that the only difference between the PDU I was crafting and the actual working PDU was the destination hardware address.

That's an amazing library @mfontanini. Great work, really!

[mfontanini]

Thanks! Sorry for not replying at all and good thing you managed to find your problem. People normally post here and expect others to solve their problems for them, so it's good to see someone putting the effort to find the issues on their own for once.

[formix]

NP, I saw that you were committing on some other things meanwhile. I thought you were a bit overwhelmed and since I'm used to take things in my own hands I continued digging. The stake was high on my side and I had to make that work!

Wish you well!