sendmsg error is too loud for a library code

[nazar-pc]

I'm observing these messages in libp2p-based app:

WARN quinn_udp: sendmsg error: Os { code: 99, kind: AddrNotAvailable, message: "Cannot assign requested address" }, Transmit: { destination: [2a0c:5a82:ea10:4900:9764:399c:4a8b:18d5]:30433, src_ip: None, enc: Some(Ect0), len: 1200, segment_size: None }

This is just IPv6 address on IPv4-only machine, which is nothing special or problematic really, it should be up to the app to log it or not. However, quinn-udp logs it automatically.

What do you think about decreasing "sendmsg error" to debug or trace level?

[Ralith]

This log message is visible so that we learn about unforeseen situations. Known-harmless errors are masked out explicitly. This error is unexpected: Quinn will only permit connection attempts to IPv6 peers if the endpoint itself is bound to an IPv6 address. If your environment is IPv4 only, then the endpoint should not have been able to bind to an IPv6 address.

What address is the endpoint bound to? What precise operation prompts the error?

Applications or even end users that aren't interested in debugging unexpected behavior have the option of configuring tracing to filter out quinn_udp logs regardless, but that has its own risks.

[nazar-pc]

My local application doesn't have IPv6 networking and doesn't have ability to connect to IPv6 nodes. However, some nodes on P2P network do have IPv6.

In libp2p nodes share addresses with each other, including sharing IPv6 addresses with IPv4-only nodes because they have no idea if those are IPv4-only or not.

My expectation is that Rust libraries only log critical errors that should never happen and are bugs, the rest should be bubbled up to application that is using the library to decide what to do with it. So far quinn-udp is the only library I consistently see logging things that are not bugs and are 100% expected.

For example here is another one that I suspect might be happening on IPv6-enabled machines:

WARN quinn_udp: sendmsg error: Os { code: 10051, kind: NetworkUnreachable, message: "A socket operation was attempted to an unreachable network." }, Transmit: { destination: [240e:390:2431:eac0:e8db:3de1:1205:e147]:30433, src_ip: None, enc: Some(Ect0), len: 1200, segment_size: None }

While I do agree it is possible to tweak log level, current situation is very annoying for two reasons:

• there might be other warnings in the library we should actually see that will not be seen if log level is changed to error for this library
• for every single other library (and there are hundreds of them in dependencies right now) info level results in very reasonable logging, quinn-udp is the only exception that every user downstream will have to become aware of and tweak to something like info,quinn_udp=error by default, which is undesirable

[Ralith]

In libp2p nodes share addresses with each other, including sharing IPv6 addresses with IPv4-only nodes because they have no idea if those are IPv4-only or not.

Okay, but that doesn't answer my questions. Quinn should not permit connection attempts from an IPv4 endpoint to IPv6 peers. Again, what address is the endpoint bound to? What precise operation prompts the error?

My expectation is that Rust libraries only log critical errors that should never happen and are bugs

As described above, this situation should never happen and looks like a bug. That said, if you only want to see errors, why isn't your log filter set to error level?

For example [ENETUNREACH] is another one that I suspect might be happening on IPv6-enabled machines

This is an interesting example. While we'd like to bubble such conditions up, it's not clear if there's a good way to do so more than we already do: failing a connection outright is probably wrong, because this condition can be transient due to a device's network configuration changing. For example, a connection attempt on a mobile device shouldn't fail immediately because you just turned off wifi and the cellular modem hasn't finished spinning up.

There's a case for logging ENETUNREACH at debug level, but this seems like exactly the sort of information you'd want a non-expert user to find in a log file if they're having connectivity issues due to an incorrectly configured environment.

[nazar-pc]

Okay, but that doesn't answer my questions. Quinn should not permit connection attempts from an IPv4 endpoint to IPv6 peers. Again, what address is the endpoint bound to? What precise operation prompts the error?

My understanding is that the following happens:

• Node A is IPv6-only
• Node B is capable of both IPv4 and IPv6
• Node C is IPv4-only
• C learns about A through B and tries to connect to it, but it can't

This is in a P2P network and I am a user of a libp2p library and utilize Quinn indirectly. So that is my understanding of the workflow, but it might be incorrect.

There's a case for logging ENETUNREACH at debug level, but this seems like exactly the sort of information you'd want a non-expert user to find in a log file if they're having connectivity issues due to an incorrectly configured environment.

Probably, I'm not yet sure about how widespread this one is, have not seen it on the machines I managed yet, but we do have other indications of network issues in P2P environment like total number of connections and other metrics, so from user's point of view printing these sporadically it non-actionable most of the time.

[sumirasystem]

same issue sendmsg error: Os { code: 99, kind: AddrNotAvailable, message: "Cannot assign requested address" }, Transmit: { destination: [2001:620:610:808:e8c2:d427:f4f8:f202]:30433, src_ip: None, enc: Some(Ect0), len: 1200, segment_size: None }

[nazar-pc]

Hey @Ralith, does my explanation above sound plausible? I'd like to help make some progress on this one in the near future.

[Ralith]

I'm interested in the answers to the specific, concrete questions I asked. Attempts to connect across address family shouldn't get this far, and we can't fix whatever issue is causing it without details. To reiterate:

• What address is the Quinn endpoint bound to?
• What precise Quinn API operation prompts the error?

I'm not categorically opposed to reducing the error severity, but it seems to me that the current severity is working as intended in bringing issues like this to our attention.

[nazar-pc]

@kpp or @mxinden, would either of you be able to help with above questions since you have way more context into how things are used in libp2p?