MacOS IPv6 UDP - No Route to Host (Works on Linux and Windows)

tmulle commented 3 years ago

Version

Which version(s) did you encounter this bug ?

Vert.x: 4.1.2

Context

I encountered an exception which looks suspicious while trying to use UDP packet sending on MacOS.

Do you have a reproducer?

Yes, I hope it is helpful. I spent a lot of time to make it configurable and to try to demonstrate the scenario I'm trying to accomplish and show you the error I am getting.

I tried to give as much documentation as possible..

https://github.com/tmulle/VertxNettyUDP6Tester

Steps to reproduce

Follow README in the repo

Extra

All information is in the README

tmulle commented 3 years ago

I've included a pure Java java.net.* version of the application as well to demonstrate that using standard Java networking works properly.

vietj commented 3 years ago

I've run your reproducer and I can observe this behaviour using the wifi interface.

When I run the reproducer with the Java version I can observe with Wireshark that although packets are sent to the wifi interface, they actually use the loopback interface.

Can you check that on your side?

tmulle commented 3 years ago

Hi Julien,

I won’t be able to work on this for a few days. I’m about to leave for a vacation until Friday.

That’s an interesting observation however.

I will look when I get back if you still need me to.

Thanks, Tim

On Aug 9, 2021, at 9:36 AM, Julien Viet @.***> wrote:

I've run your reproducer and I can observe this behaviour using the wifi interface.

When I run the reproducer with the Java version I can observe with Wireshark that although packets are sent to the wifi interface, they actually use the loopback interface.

Can you check that on your side?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

vietj commented 3 years ago

thanks, I would like just to see if we are in the same conditions.

I suspect that the packet does not get routed on OSX in my case because my router does not handle IPV6, hence it makes sense to fail.

On Mon, Aug 9, 2021 at 3:48 PM tmulle @.***> wrote:

Hi Julien,

I won’t be able to work on this for a few days. I’m about to leave for a vacation until Friday.

That’s an interesting observation however.

I will look when I get back if you still need me to.

Thanks, Tim

On Aug 9, 2021, at 9:36 AM, Julien Viet @.***> wrote:

I've run your reproducer and I can observe this behaviour using the wifi interface.

When I run the reproducer with the Java version I can observe with Wireshark that although packets are sent to the wifi interface, they actually use the loopback interface.

Can you check that on your side?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/eclipse-vertx/vert.x/issues/4059#issuecomment-895237716, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABXDCQD5AQW55S72DZIGZDT37MBPANCNFSM5BWKBEIA .

tmulle commented 3 years ago

Hi Julien,

I have updated my tester code for the regular java version so that it uses the specified interface when sending the UDP multicast traffic. I accidentally left off the part to set the interface so it was defaulting to the loopback interface. I saw the same thing in Wireshark as you did.

I also updated my Vertx version of the Sender class so that it also sets the Multicast interface when sending out the data and that seems to fix the sending of multicast UDP packets.

However, sending plain non-multicast UDP packets in the Vertx version still gives me the same "No Route to Host" error as before.

tmulle commented 3 years ago

Just wondering is there anymore information on this topic?

Is there any more testing you'd like me to do? Was my tester program sufficient?

Thanks..

tmulle commented 3 years ago

FYI.. I just tried my tester code with 4.2.0.Beta1 and still the same issue with IPv6

vietj commented 3 years ago

I think we could try checking why it is not routed when it uses the non loopback interface and see if adding a multicast routing rule would make it work

vietj commented 3 years ago

that what I was suspecting previously afair

tmulle commented 3 years ago

Is there anymore information on this? Anything I can look at on my side? I notice this was now pushed into 4.3 Anything from the Netty folks?

tmulle commented 3 years ago

Update: If I run my code using JDK17 with both the Sender and Receiver on the same host, things work fine with UDP IP6. However, if I switch back to JDK11 try both the Sender and Receiver on the same host, I get the error as before.

Trying the Sender on a linux machine and my Receiver (Device Mode) on my Mac, still fails under both JDKs when trying to respond back to the sender UDP address and port.

vietj commented 3 years ago

this seem to be a JDK issue, we should close this issue.

tmulle commented 3 years ago

Well, it still doesn't work when try to send a response back to a remote system (A receiver on another machine). In JDK17 if you run both the sender and receiver on the same machine things work fine. But as soon as you separate the processes, even JDK17 still throws the same error when using the Vert.x/Netty code. Using the regular Java Network API on both JDK11 or JDk17, with the processes on both or remote machines works fine. So, it still feels like there is something in Netty/Vert.x.

You can see what I mean with my test code.

https://github.com/tmulle/VertxNettyUDP6Tester

Local Machine

Run two processes: A sender and Device mode on the same machine but in two different shells. Under JDk11 the Device mode will fail when trying to send back the response to the sender. Under JDK17, things work fine

Remote Machine

Run two processes: A sender and Device mode on two different machines on the network Under JDk11 and JDK17 the Device mode will fail when trying to send back the response to the sender.

Run the same tests with the Regular java test code I wrote and things work fine regardless of the JDK used.

denniszheng7 commented 3 years ago

Hi: I've tested your tester code on my Mac, but the test result is all right whether use Java NIO classes or Vertx code in IPv6 mode, here is my test result:

[main] INFO netty.udp.VertUDPTester - *** Starting Variables ***
[main] INFO netty.udp.VertUDPTester - Operation Mode = Both
[main] INFO netty.udp.VertUDPTester - Network Mode = IPv6
[main] INFO netty.udp.VertUDPTester - Network Interface = en0
[main] INFO netty.udp.VertUDPTester - Network Listen All Interface = ::
[main] INFO netty.udp.VertUDPTester - Network Multicast Group = FF02::1
[main] INFO netty.udp.VertUDPTester - Network Port = 35056
[main] INFO netty.udp.VertUDPTester - **************************
[vert.x-eventloop-thread-0] INFO netty.udp.VertUDPTester$Sender - Listening for Unicast message on IPv6 @ 0:0:0:0:0:0:0:0:56094
[vert.x-eventloop-thread-1] INFO netty.udp.VertUDPTester$Device - Listening for Multicast Messages on IPv6 Group [FF02::1] on local address (en0): 0:0:0:0:0:0:0:0:35056
[vert.x-eventloop-thread-0] INFO netty.udp.VertUDPTester$Sender - Starting to send Discovery Requests every 10 seconds
[vert.x-eventloop-thread-0] INFO netty.udp.VertUDPTester$Sender - Sending request...
[vert.x-eventloop-thread-0] INFO netty.udp.VertUDPTester$Sender - Request sent successfully
[vert.x-eventloop-thread-1] INFO netty.udp.VertUDPTester$Device - Received Discovery Packet from fe80:0:0:0:1c39:274:2260:9983%6:56095 with data {"action":"REQUEST","responsePort":56094}
[vert.x-eventloop-thread-1] INFO netty.udp.VertUDPTester$Device - Sending response back to fe80:0:0:0:1c39:274:2260:9983%6:56094 with data - {"id":"9520fdfa-e6e4-4f7d-9b7e-974d6cf83e9d","date":"2021-11-18T09:11:40.288061"}
[vert.x-eventloop-thread-0] INFO netty.udp.VertUDPTester$Sender - Got a discovery response message from fe80:0:0:0:1c39:274:2260:9983%6:35056 with data - {"id":"9520fdfa-e6e4-4f7d-9b7e-974d6cf83e9d","date":"2021-11-18T09:11:40.288061"}
[vert.x-eventloop-thread-1] INFO netty.udp.VertUDPTester$Device - Response Sent ok
[vert.x-eventloop-thread-0] INFO netty.udp.VertUDPTester$Sender - Sending request...
[vert.x-eventloop-thread-0] INFO netty.udp.VertUDPTester$Sender - Request sent successfully
[vert.x-eventloop-thread-1] INFO netty.udp.VertUDPTester$Device - Received Discovery Packet from fe80:0:0:0:1c39:274:2260:9983%6:56095 with data {"action":"REQUEST","responsePort":56094}
[vert.x-eventloop-thread-1] INFO netty.udp.VertUDPTester$Device - Sending response back to fe80:0:0:0:1c39:274:2260:9983%6:56094 with data - {"id":"9520fdfa-e6e4-4f7d-9b7e-974d6cf83e9d","date":"2021-11-18T09:11:50.229602"}
[vert.x-eventloop-thread-1] INFO netty.udp.VertUDPTester$Device - Response Sent ok
[vert.x-eventloop-thread-0] INFO netty.udp.VertUDPTester$Sender - Got a discovery response message from fe80:0:0:0:1c39:274:2260:9983%6:35056 with data - {"id":"9520fdfa-e6e4-4f7d-9b7e-974d6cf83e9d","date":"2021-11-18T09:11:50.229602"}
Disconnected from the target VM, address: '127.0.0.1:59406', transport: 'socket'

I have the same Java version as you use.

openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)

My system is a 2020 13" Macbook Pro running Catalina (10.15.5) and use Wireless networking. Wireless Network Settings

en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=400<CHANNEL_IO>
    ether 90:9c:4a:b6:bf:5c
    inet6 fe80::1c39:274:2260:9983%en0 prefixlen 64 secured scopeid 0x6
    inet6 2408:8456:3244:314a:7b:59e3:3ba9:4775 prefixlen 64
    inet 192.168.43.71 netmask 0xffffff00 broadcast 192.168.43.255
    nd6 options=201<PERFORMNUD,DAD>
    media: autoselect
    status: active

Furthermore, when i execute the code first time, system firewall promts something like "Are you allowed to pass" and select "Allow", but i use wireshark for packet capture, it shows that no UDP packets are coming in through the en0 interface.

tmulle commented 3 years ago

Interesting, I've updated the code to use Vertx 4.2.1 and I still get the issue. I've also fixed the service_mode parameter to be net.service_mode like the Readme says. I also added profiles for JDK8,11, and 17.

However, interesting finds:

IPv6 Mode

JDK17 works fine if both Sender and Device running on same machine - Fails once they are separated on two different machines
JDK8 and JDK11 fail with the error even if both Sender and Device mode running on same machine or split up.

IPv4 Mode

All JDKs and combinations of local and remote Sender and Device modes work properly

The Regular Java version always works for me no matter how I run it.

tmulle commented 2 years ago

Any further ideas on this? Am I the only one using VertX UDP in this manner?

tmulle commented 2 years ago

@vietj This is fixed in Netty 4.1.74.Final. I tested by overriding the Netty dependencies in Vert.x to use version 4.1.74.Final and the issue is resolved.

Just letting you all know..that once Vert.x updates its Netty dependencies to 4.1.74.Final things will work for me.

vietj commented 2 years ago

thanks for checking that, can you link the corresponding netty issue ? we will upgrade to 4.1.74.Final

tmulle commented 2 years ago

@vietj Here's the link to the Netty issue I reported that is fixed.

https://github.com/netty/netty/issues/11563

vietj commented 2 years ago

I can see good investigation from your side in the Netty project !!!

On Wed, Feb 9, 2022 at 4:57 PM tmulle @.***> wrote:

@vietj https://github.com/vietj Here's the link to the Netty issue I reported that is fixed.

netty/netty#11563 https://github.com/netty/netty/issues/11563

— Reply to this email directly, view it on GitHub https://github.com/eclipse-vertx/vert.x/issues/4059#issuecomment-1033914725, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABXDCVRJ5CNCVJVHHTZFS3U2KFFRANCNFSM5BWKBEIA . You are receiving this because you were mentioned.Message ID: @.***>

eclipse-vertx / vert.x