search

NICMx / Jool

SIIT and NAT64 for Linux
GNU General Public License v2.0
328 stars 66 forks source link

Stateful NAT64 pool6-only with changing IPv4 address #216

Closed arurke closed 8 years ago

arurke commented 8 years ago

I have a gateway which has one IPv6 interface and one or more IPv4 interfaces. On IPv6 side I have nodes transmitting UDP traffic every sec. This goes through the gateway with Jool stateful NAT64 and only pool6 configured:

modprobe jool pool6=64:ff9b::0:0/96

This works fine until the address on the IPv4 interface changes (or another interface become the preferred route). In this case Jool will maintain the old IPv4 mapping even in the case where this IP is not on any interface anymore (e.g. IPv4 interface changed IP). I would expect Jool to create a new mapping (especially if the IP is not longer valid) if it finds a better IPv4, and let the old mapping time out.

Issuing "jool -4 -f" will solve the situation. Or seizing the transmission from the IPv6 node for the session expiration timer and then resuming.

Is this a bug? Limitation? Something I am misunderstanding?

Running Jool 3.4.3, Beaglebone Black with debian 4.1.17-ti-rt-r48.

ydahhrk commented 8 years ago

You want two mappings (BIB entries) to have the same client at the same time. RFC 6146 expects each mapping's IPv6 client to be a primary key for the mapping:

A given IPv6 or IPv4 transport address can appear in at most one entry in a BIB

(page 14)

So I'd say this is a limitation of NAT64 itself.

That said, the prospect of being able to change the pool4 address without disrupting the traffic is attractive.

As long as Jool is in pool4 empty mode, it would not be that difficult for it to realize that the interface IPv4 address has changed and adapt the whole BIB to it. Though this might lead to catastrophic results (performance and/or functionality-wise) if there is more than one IPv4 interface and therefore more than one IPv4 address.

jool -4f will cause all mappings to be recomputed, which can lead to different IPv4 port masks being assigned to the IPv6 clients. Is this not a problem? Perhaps it would be cleaner if we would provide a userspace means to change all mappings matching a certain IPv4 address into a certain other IPv4 address.

Am I making sense?

arurke commented 8 years ago

Thanks for your reply, and yes you are making sense!

I wonder about the transmissions which are repeatedly refreshing the session causing this to be an infinite faulty state. As I understand the multiple IPv4 interface scenario foils the adaptive solution you mention. But instead of adapting, could you compare the bib with the new pool4 - if the IPv4 in a mapping is not found in the pool4 anymore could you safely purge this mapping? (assuming jool will never receive incoming traffic matching this mapping anyways since the interface/ipv4-address is gone and not merely moved to another interface or similar)

Regarding jool -4f. The situation is already such that the translations are "invalid" using a non-existing IPv4 address, so in this scenario I have no traffic flowing and whatever consequences from flushing are to the better. Your suggestion would preserve the port, but the IPv4 would naturally be new and would this not have the same consequence for any users?

ydahhrk commented 8 years ago

if the IPv4 in a mapping is not found in the pool4 anymore could you safely purge this mapping?

Yes, it's doable.

Although this is kind of the antithesis of an old feature request, where orphan mappings were (sort of) desired. But nothing is stopping us from offering strong anti-orphaning as an alternate operation mode.

Ok so if you can confirm it can be useful (and I'm not misunderstanding), this issue would be a feature request for a global configuration flag that would behave as follows:

Your suggestion would preserve the port, but the IPv4 would naturally be new and would this not have the same consequence for any users?

Yes, the IPv4 client/server would somehow need to be aware of the address change, which indeed is pretty far-fetched (AFAIK). I was just thinking out loud.

ydahhrk commented 8 years ago

TCP:

TCP exchange

(Include orange squares when the nodes are not actively talking. Exclude orange squares when nodes are actively talking.)

UDP:

UDP exchange

arurke commented 8 years ago

My setup has a 3G modem which gets a new IPv4 address now and then, so its useful for me, and maybe all other with similar devices.

As you point out in your sketches there will be issues (I only use UDP) but in any case I would argue it is better than the alternative (nothing working).

"Adds an extra lookup", would this give any significant performance penalty?

ydahhrk commented 8 years ago

"Adds an extra lookup", would this give any significant performance penalty?

When pool4 is empty, the lookup is in a single node list, so no :)

arurke commented 8 years ago

Would this extra lookup not be executed on every translation of each frame? Also in normal operation (during blah blah) when pool4 has entries? (maybe I am not fully understanding pool4 in this case)

ydahhrk commented 8 years ago

Would this extra lookup not be executed on every translation of each frame?

It would, but it's no big deal since the list is so small.

Also in normal operation (during blah blah) when pool4 has entries?

When pool4 has entries, the address is not expected to change randomly, so I would expect --avoid-orphans to be disabled.

The feature will be disabled by default, so I don't expect the extra lookup to get in the way unless the admin is really clumsy.

It is an interesting question though, as it raises another one: Why do the need the flag at all? We could always include the lookup when pool4 is empty, and always exclude it when pool4 has entries.

I suppose that's the right solution.

arurke commented 8 years ago

:+1:

arurke commented 8 years ago

Btw, I just noticed on the FAQ it say: "Stateful Jool’s minimum configuration requirements are

arurke commented 8 years ago

Kind of hijacking this thread for some troubleshooting, but it might be relevant for the issue to clarify my understanding of pool4:

As I mentioned, my config is:

modprobe jool pool6=64:ff9b::0:0/96

I have one IPv6 node TXing every sec, so my --session looks like this:

~$ sudo jool --session
UDP:
---------------------------------
Expires in 4 minutes, 59 seconds
Remote: 179.12.213.22#5683     fd51:543b:940e:0:214:4b00:a28:fe22#5683
Local: 10.1.0.12#61295             64:ff9b::b11c:eb0a#5683
---------------------------------

When the server 179.12.213.22 replies, I noticed I was getting "Port unreachable" in return. So I compiled with DDEBUG and found the frame from the server:

Catching IPv4 packet: 179.12.213.22->10.1.0.12
Step 1: Determining the Incoming Tuple
Tuple: 179.12.213.22#5683 -> 10.1.0.12#61295 (UDP)
Done step 1.
Step 2: Filtering and Updating
Packet does not belong to pool4.
Returning the packet to the kernel.

"does not belong to pool4". And my pool4:

~$ sudo jool --pool4
  (empty)

Now this is when I started to doubt my understanding of pool4. Is this correct behavior? I would expect jool to match the incoming frame to the session, and translate. I.e. pool4 would contain of whatever dynamically added IPv4 address#port jool has used (maybe not actually, but in practice). If not, the configuration without pool4 does not allow incoming traffic?

ydahhrk commented 8 years ago

"At least one prefix/address in the IPv4 pool." Legacy?

Yes, thank you!

I'll get that removed asap.

Is this correct behavior? I would expect jool to match the incoming frame to the session, and translate. I.e. pool4 would contain of whatever dynamically added IPv4 address#port jool has used (maybe not actually, but in practice). If not, the configuration without pool4 does not allow incoming traffic?

I think your understanding of pool4 is correct; Jool is misbehaving. We are looking at either a bug or a misconfiguration.

This is what I can see can go wrong (and which can lead to Jool thinking that the packet is not translatable) when Jool checks whether the address "belongs" to an empty pool4:

  1. The port (61295) is less than 61001. This is clearly not the case.
  2. The port (61295) is higher than 65535. This is clearly not possible.
  3. The code cannot find 10.1.0.12 when it looks for a "universe"-scope address across the IPv4 interfaces.
    This is likely the problem.

The intent of that (third) check is to prevent Jool from picking an address that might be actually link-scoped (ie. unable to leave the network), and thus unsuitable for traffic intended to reach the Internet. Perhaps the fact that your address is temporary means that it is somewhat below "universe"-scoped, but it's weird because if it was chosen during the 6-to-4 translation then it was universe-scoped then. So maybe it is arbitrarily changing scopes every now and then?

Sorry that I'm asking you to get into Jool's entrails, but perhaps you might want to scatter more debug messages to pinpoint the exact source of the problem. This should get you started:

Here are the definitions of scope. Notice that there is room for intermediary, user-defined values.

Here's an example of a debug message:

To debug the exact address and port being looked up by pool4db_contains(), inject this in pool4_contains() (after the variable declarations and before found = pool4empty_contains(addr);):

log_debug("The transport address I'm looking for is %pI4#%u", &addr->l3, addr->l4);
arurke commented 8 years ago

Hi, thanks a lot, I sprinkled the code with some debug messages and think I found the problem.

In contains_addr() you are comparing using ifa->ifa_address. Printing out this value I noticed that for point-to-point interfaces it does not contain the local address. Excerpt from ifconfig and some debug messages from for_each_netdev_rcu() in contains_addr() to illustrate:

~$ sudo ifconfig

eth0      inet addr:10.0.0.1  Bcast:10.0.0.3  Mask:255.255.255.252

ppp0      Link encap:Point-to-Point Protocol  
          inet addr:10.253.191.133  P-t-P:10.64.64.64  Mask:255.255.255.255

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:10.8.0.14  P-t-P:10.8.0.13  Mask:255.255.255.255
Starting contains_addr() looking for:  10.8.0.14

Are now checking out ifa with ifa_address: 127.0.0.1 and ifa_local: 127.0.0.1
Are now checking out ifa with ifa_address: 10.0.0.1 and ifa_local: 10.0.0.1
Are now checking out ifa with ifa_address: 192.168.7.2 and ifa_local: 192.168.7.2
Are now checking out ifa with ifa_address: 10.64.64.64 and ifa_local: 10.253.191.133
Are now checking out ifa with ifa_address: 10.8.0.13 and ifa_local: 10.8.0.14

So ifa_address actually contains the point-to-point address instead of the local interface address. See http://lxr.free-electrons.com/source/include/uapi/linux/if_addr.h?v=3.10#L16.

I changed to using ->ifa_local instead and it solved my problem. I will open a separate issue on it. If you want I can submit my changes (in empty.c and blacklist4.c) in a PR. (note I have not gone deeper into if_addr.c/h to see if this has other implications).

ydahhrk commented 8 years ago

Ok, this feature request was implemented and is now available in the test branch.

When pool4 is empty, Jool now makes sure the relevant BIB entry is still valid during every translation. This is achieved through a constant-time operation (a lookup in a guaranteed single-node list) so it does not add significant overhead.

@arurke: Mind testing whether this does exactly what you wanted? Though the commit is 3.5 code, the new features should not change the way you manage the translator. (as long as you're not using them.)

arurke commented 8 years ago

Works! 👍 Should I close or wait for 3.5?

ydahhrk commented 8 years ago

Whatever; we normally wait till the code is collapsed into master, but issues opened by users are their own.

arurke commented 8 years ago

I'll do it like this, or I will forget it ;) thx for the fix