New map.sh (March 5) Important Bugfix, New Feature to not SNAT to certain ports.

[cmspam]

In my previous map.sh files, there were some various bugs with the nftables rules. We also resorted to fwmarks to solve the problem.

I also wanted to add a feature for people who run some servers or have some open ports, not to SNAT to those ports.

After doing some studying about nftables, I found a much better way to handle everything.

Bugfix:

1. My use in previous map.sh files of the numgen feature of nftables was flawed, and many packets were not being marked correctly by the rules. I have completely redesigned my use of the feature, and have tested thoroughly, and everything is now working correctly.

Here are the differences:

1. We no longer need to mark any packets. (YAY!) Now there will be zero conflict with anything that marks packets.
2. We now SNAT the packets sequentially by port. You will see the usable ports, being used in order if you run tcpdump for instance.
3. Modifications are clearly marked in the map.sh file with big comments.
4. You can now set the DONT_SNAT_TO variable at the beginning of the file with a list of ports that you don't want to SNAT to. I suggest if you run a server of some kind on some port, you add it to this list and don't SNAT to it.

I feel like this is finally a complete solution. As before, only the map.sh file needs to be modified. Because there were many bugs that prevent connections from being made, I ask that fakemanhk please update the file used to this one, so that other people don't have the issues my sloppiness introduced when I was first figuring it out. map.sh.20230305.txt

[TimOfAllTrades]

Tested and it works. Thanks again.

[cmspam]

Tested and it works. Thanks again.

Thanks for confirming. Can you please let me know which provider you're using?

[TimOfAllTrades]

Tested and it works. Thanks again.

Thanks for confirming. Can you please let me know which provider you're using?

Docomo Hikari with OCN provider. No Hikari Denwa so /64 prefix. I believe they also use some implementation of MAPE

[cmspam]

Thanks! OCN uses more port ranges and slightly different settings than JPNE which I use, so I'm happy it works as expected on both.

[fakemanhk]

Got it working now, even the suumo.jp website is loading properly without issue, pasting the diff between stock map.sh vs new version here:

root@OpenWrt# diff -c map.sh.old map.new
*** map.sh.old  Sat Mar  4 03:57:30 2023
--- map.new     Tue Mar  7 00:09:22 2023
***************
*** 13,18 ****
--- 13,40 ----
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.

+ #------------------------------------
+ #     Modifications by 'cmspam'
+ #------------------------------------
+ # These modifications are made to ensure that the multiple port ranges
+ # used with Japan's MAP-E implementation are actually SNAT-ed to correctly.
+ 
+ #------------------------------------
+ #     Setting to not SNAT to certain ports
+ #------------------------------------
+ # This is a space-delimited list of ports to not SNAT to.
+ # If, for example, you use some ports for servers, it's best to
+ # include them here, so that they are not used with SNAT.
+ # Port ranges are not supported, so please list each port individually
+ # separated with a space.
+ # 
+ # Example, if you want to not SNAT to 2938, 7088, and 10233
+ #
+ #DONT_SNAT_TO="2938 7088 10233"
+ 
+ DONT_SNAT_TO="0"
+ 
+ 
 [ -n "$INCLUDE_ONLY" ] || {
       . /lib/functions.sh
       . /lib/functions/network.sh
***************
*** 140,158 ****
             json_add_string snat_ip $(eval "echo \$RULE_${k}_IPV4ADDR")
           json_close_object
         else
           for portset in $(eval "echo \$RULE_${k}_PORTSETS"); do
!               for proto in icmp tcp udp; do
!               json_add_object ""
!                 json_add_string type nat
!                 json_add_string target SNAT
!                 json_add_string family inet
!                 json_add_string proto "$proto"
!                   json_add_boolean connlimit_ports 1
!                   json_add_string snat_ip $(eval "echo \$RULE_${k}_IPV4ADDR")
!                   json_add_string snat_port "$portset"
!               json_close_object
!               done
           done
         fi
         if [ "$maptype" = "map-t" ]; then
               [ -z "$zone" ] && zone=$(fw3 -q network $iface 2>/dev/null)
--- 162,233 ----
             json_add_string snat_ip $(eval "echo \$RULE_${k}_IPV4ADDR")
           json_close_object
         else
+ 
+ #------------------------------------
+           #MODIFICATION 1: Get all ports, and full port count.
+           #                Don't include ports in DONT_SNAT_TO
+ #------------------------------------
+           local portcount=0
+           local allports=""
           for portset in $(eval "echo \$RULE_${k}_PORTSETS"); do
!               local startport=$(echo $portset | cut -d'-' -f1)
!               local endport=$(echo $portset | cut -d'-' -f2)
!               for x in $(seq $startport $endport); do
!                       if ! echo "$DONT_SNAT_TO" | tr ' ' '\n' | grep -qw $x; then                        
!                               allports="$allports $portcount : $x , "
!                               portcount=`expr $portcount + 1`
!                       fi
!               done
           done
+           allports=${allports%??}
+ #------------------------------------
+           #END MODIFICATION 1
+ #------------------------------------
+           
+ #------------------------------------
+             #MODIFICATION 2: Create mape table
+ #------------------------------------
+             nft add table inet mape
+             nft add chain inet mape srcnat {type nat hook postrouting priority 0\; policy accept\; }
+ #------------------------------------
+           #END MODIFICATION 2
+ #------------------------------------
+ 
+ 
+ #------------------------------------
+           #MODIFICATION 3: Create the rules to snat to all the ports
+ #------------------------------------
+           local counter=0
+           
+             for proto in icmp tcp udp; do
+                 nft add rule inet mape srcnat ip protocol $proto oifname "map-$cfg" snat ip to $(eval "echo \$RULE_${k}_IPV4ADDR") : numgen inc mod $portcount map { $allports }
+             done
+           #END MODIFICATION 3
+           
+ #------------------------------------
+           #MODIFICATION 4: Comment out original SNAT implementation.
+ #------------------------------------
+ #            for portset in $(eval "echo \$RULE_${k}_PORTSETS"); do
+ #              for proto in icmp tcp udp; do
+ #                json_add_object ""
+ #                  json_add_string type nat
+ #                  json_add_string target SNAT
+ #                  json_add_string family inet
+ #                  json_add_string proto "$proto"
+ #                  json_add_boolean connlimit_ports 1
+ #                  json_add_string snat_ip $(eval "echo \$RULE_${k}_IPV4ADDR")
+ #                  json_add_string snat_port "$portset"
+ #                json_close_object
+ #              done
+ #            done
+ #------------------------------------
+           #END MODIFICATION 4
+ #-------------------------------------
+ #------------------------------------
+           #ALL MODIFICATIONS FINISHED
+ #------------------------------------
+ 
+ 
         fi
         if [ "$maptype" = "map-t" ]; then
               [ -z "$zone" ] && zone=$(fw3 -q network $iface 2>/dev/null)

[fakemanhk]

Confirmed this is working now, updated the file with new content.