LiveTraffic sending data over wrong NIC

Eggman1414 commented 7 months ago

Describe the bug

When used in a master client configuration with external views (each running XPMP2-Remote), LiveTraffic (Master) sends out the position data for the remote clients on the first NIC listed in windows. When first NIC is disabled, traffic is seen on clients.

Expected behavior

Send multicast data over same nic used for external view configuration. Or have a drop down selection for which NIC to use for sending data.

Technical Info

OS: Win
X-Plane version: 12.07
LiveTraffic version: 3.4.3

Log.txt

TwinFan commented 7 months ago

Similar to TwinFan/LiveTraffic#221. Totally different functionality, but behind the scenes similar networking code at play.

TwinFan commented 7 months ago

The "Remote" functionality uses UDP Multicast. Reading through some Stackoverflow threads I believe the receiving side of Mutlicast is fine with binding to INADDR_ANY and will listen on all available interfaces. But not the sending side as this reply makes very clear. Sending seems to happen over one interface only, even when bound to INADDR_ANY.

Also see section 6.3 IP_MULTICAST_IF in this document, offering a way to specify the sending interface via a call to setsockopt, and it looks like this is possible any time.

The basic idea to solve this is "to send to where we received from", making use of the fact that also the Remote Client sends out a Beacon if no actual traffic data is yet broadcasted:

[X] While only sending out beacons, send out beacons on all interfaces. See here for example how to get a list of all interface (Linux/Mac...Windows different as always)
[X] Once we received any data (like a beacon, but also actual traffic) from any interface, send only on that interface
[x] Add a config option to the XPMP2.prf to define the sending interface as a last resort

TwinFan commented 7 months ago

@Eggman1414 . would you want to try updated software? Here's a beta version of LiveTraffic together with an updated version of the Remote Client. "Beta" means time-limited for one month, but I plan to publish official versions long before that. Two things yet missing, one is a configuration option to tell the remote functionality the interface to use. But I'd actually appreciate feedback on this version without the config option because I try hard to keep the Remote Client a zero-config software: It shall auto-discover the correct network interface to use. For that both sender (LiveTraffic) and receive (Remote Client) need to be updated.

For more details refer to the two readme.html files included for both plugins. You will see that LiveTraffic has a lot more updates, for which documentation isn't yet fully available, but our focus shall lie on the remote functionality anyway.

From your issue I guess you are on Windows, only Windows included here: Remote_BETA_v340.zip

Thanks in advance for any feedback! In case of any issue please add Log.txt files from both sender and receiver with log level set to "Debug".

Eggman1414 commented 7 months ago

@TwinFan Sure thing! I will test tomorrow and let you know the results. Thank you

Eggman1414 commented 7 months ago

@TwinFan Installed the new build and same issue. I have included both logs (1 from Master and 1 from one of the Clients (there are 4 in total). Looks like clients are sending beacon back correctly over multicast but not Master to Client. Everything should be going out DIS NIC.

Master - 10.100.40.100 Client - 10.100.40.101 Log-Master.txt Log-Client.txt

TwinFan commented 7 months ago

Hm, that's a pitty given all the effort put into that code... During discovery, the Client is to send a beacon, the Master is just to listen, there is no traffic yet expected from the master until a client's datagram reached it on any interface. We would see it in the logs once the first beacon reaches the Master...it didn't.

In the Wireshark screenshot

the top right is likely a beacon sent by the Remote Client (8 bytes on the correct port 49788). Given that you say the master (also) has address 10.100.40.100 I am wondering why that beacon datagram wouldn't reach the master? You don't happen to have some kind of virtualization interface running, like from a VMWare Client/Player? (That was recently a reason why in my setup multicasts wouldn't be transferred but kind of swallowed but that interface.)
the bottom left shows a datagram by X-Plane on port 49707. Are any of the other datagrams on that interface sent from/to port 49788, ie. by the Remote Client?

Latest Build

Anyway...I provide you here with the latest build: Remote_BETA_v340a.zip

This adds a little more output for the log: It will show which interfces the clients are aware of. And it now also includes the possibility to explicietly specify the interface the remote functionality is to use only, meant as a last resort.

Without Config File

Please try the new version once again without config file and let's check the output on identified interfaces. Should look something like this:

0:18:39.556    LT/XPMP2 DEBUG Remote.cpp:685/RmtSendMain: Sender thread started
0:18:39.556    LT/XPMP2 DEBUG Network.cpp:866/Join: MC 239.255.1.1:49788: Joined INADDR_ANY, known interfaces: lo0,en0
0:18:39.556    LT/XPMP2 INFO  Remote.cpp:709/RmtSendMain: Listening to 239.255.1.1:49788, waiting for someone being interested in our data...
...
0:18:39.765    LT/XPMP2 INFO  Network.cpp:969/SendToIntf: MC 239.255.1.1:49788: Sending via interface 'en0' only
0:18:39.765    LT/XPMP2 INFO  Remote.cpp:740/RmtSendMain: Received word from 192.168.178.55:49788, will start sending aircraft data

Specifying the Interface to Use

I don't expect it to work, though, nothing should have changed in that respect. Then go ahead and configure the interface to use: Copy the XPMP2.example.prf file to <X-Plane>/Output/preferences and rename it to XPMP2.prf. In the file you'll find detailed usage instructions, and the new key remoteSendIntf. Remove the leading # to activate a line and name the interface to use, like

remoteSendIntf Ethernet

Do that on both Master and Client. (Obviously, the configured interface could be named differently on those two hosts, but both hosts need that XPMP2.prf config file with their respective interface configured.) And then give it another try...

Eggman1414 commented 7 months ago

Ok I think I might of found something. Im pretty sure XPMP2-Remote is working fine. I am receiving the beacon packets from Client(10.100.40.101) on Master(10.100.40.100). Packet: 239.255.1.1 and Port 49788. But it seems like LiveTraffic is not responding to the requests.

On the master PC I have 3 NIC's (1 for Sim Traffic, 1 for Avionics Traffic, and a Docker NIC). If I disable the First NIC (Avionics), and reload the plugin, BAM!. Traffic is now being sent by LiveTraffic on Master to Clients.

So in short, XPMP2-Remote is probably fine, but LiveTraffic might be having problems with multiple nics.

Eggman1414 commented 7 months ago

Adding XPMP2.prf to Master Client and Changing remoteSendIntf to "remoteSendIntf DIS" worked on launch. Did not have to change Client (But Im guessing because the client only has one interface so not an issue)

TwinFan commented 7 months ago

Thanks for trying it. So the specific config did its intended work, great. And, yes, no need to configure the client when it just has one NIC. I can't tell you just now why LiveTraffic doesn't receive the Beacon. The code is literally the same as for the client, it is just one library used for both master and client functionality. I would have liked to see the log from a start of the latest version of LiveTraffic without the config file because it would tell us in the Network.cpp:866/Join line which NICs it sees. But with IPv4 I have it just join the ANY address, I don't join specific interfaces. That is supposed to work fine, and does in my setups, but apparently not in yours. (With IPv6 listening to the ANY address no longer listens on all interfaces anyway and instead I have the socket join all interfaces in a loop...so switching to IPv6 would probably lead different results.) With the config file, the socket joins specifically the given interface, not just generically ANY, so it is more specific. Maybe I can cover your setup ("issue") by also having the IPv4 code join all interfaces specifically instead of just ANY...would even simplify the code as both IPv4 and IPv6 would fall in line...maybe I try that, can probably solve issues with other users, too. Thanks again for your patience...maybe I do one more version for you ;-)

TwinFan commented 7 months ago

@Eggman1414 , would you do me the favour to try these versions without using a XPMP2.prf config file? Strictly speaking, updating LiveTraffic on the master and removing XPMP2.prf should suffice. Remote_BETA_v340b.zip

With LiveTraffic now explicetely joing all interfaces I have the hope that specifying the interface is no longer required. In DEBUG mode, LiveTraffic will state which interfaces it joined like this:

DEBUG Network.cpp:931/Join: MC 239.255.1.1:49788: Joined on interfaces lo0,en0

But if still LiveTraffic doesn't receive the beacon and hence doesn't start sending, then XPMP2.prf with remoteSendIntf DIS should still do the trick again.)

Eggman1414 commented 7 months ago

@TwinFan looks like that revision works without setting the XPMP2.prf on the Master. On a side note, it also seems to be working fine on version 12.1. I included the log file just incase you wanted to take a look. Thank you for looking into this issue. Log.txt

TwinFan commented 7 months ago

Great to hear, thanks for verifying! The interesting parts of the log are these:

0:01:56.185    LT/XPMP2 DEBUG Network.cpp:888/Join: MC 239.255.1.1:49788: Joined on interfaces Loopback Pseudo-Interface,DIS,AVIONICS,vEthernet (WSL)
0:01:56.185    LT/XPMP2 INFO  Remote.cpp:709/RmtSendMain: Listening to 239.255.1.1:49788, waiting for someone being interested in our data...
...
0:02:10.530    LT/XPMP2 INFO  Network.cpp:989/SendToIntf: MC 239.255.1.1:49788: Sending via interface 'DIS' only
0:02:10.530    LT/XPMP2 INFO  Remote.cpp:740/RmtSendMain: Received word from 10.100.40.102:49788, will start sending aircraft data

TwinFan / XPMP2