search

owenthewizard / opnatt

Use your own OPNsense router with AT&T Fiber [GPON].
54 stars 6 forks source link

readme

Disclaimer

Bypassing the AT&T Residential Gateway does NOT allow for theft-of-service.

Service authentication is NOT performed by the AT&T Residential Gateway (likely the ONT instead)[citation needed]. This repository does NOT contain ANY code or any other content that is proprietary, patended, copyrighted, or otherwise belonging to AT&T, Arris, Motorola, or other companies related to the AT&T Residential Gateway.

About

This repository includes notes on using your own OPNsense hardware with AT&T Fiber. The method described here uses wpa_supplicant to authenticate via 802.1x (EAP-TLS), and either netgraph or a smart/managed switch to deal with VLAN ID 0. This method eliminates the need for the AT&T Residential Gateway entirely. The ONT is still used, and plugs directly into OPNsense.

Compatibility

This repository aims to support the latest stable version of OPNsense. Compatibility with older versions or other operating systems (pfSense, Linux, etc.) is not guaranteed. That said, netgraph is the only OS-specific part of the stack. Using a smart/managed switch instead of netgraph, eBPF (Linux), or another solution should work on other operating systems.

OPNsense

The latest OPNsense release this script was tested on is OPNsense 23.1.5_4-amd64.

GPON & XGS-PON

This type of bypass does not work in XGS-PON areas.

Switches

Manufacturer Model Working HW Version SW Version Notes Updated
D-Link DGS-1100-08 ✅ B1 1.00.b031 @gpz1100 04/08/2023
D-Link DGS-1100-05v2 ✅ A1 1.00.003 @topsecretsauce 04/12/2023
TP-Link TL-SG108E ✅ 5.0 1.0.0 Build 20191021 Rel.53360 @owenthewizard 04/13/2023
D-Link DGS-1210-10 ✅ F1 6.31.002 @gpz1100 04/14/2023
Netgear GS308T ❌ @gpz1100 04/08/2023
Cisco Catalyst 3750G ❔
Mikrotik RB4011 ❔
Ubiquiti USW-Flex-Mini ❌ @bigjohns97 04/14/2023

Background

While many AT&T Residential Gateways offer something called IP Passthrough, it does not provide the same advantages of a true bridge mode. For example, the NAT table is still managed by the gateway, which is limited to a measly 8192 sessions (although it becomes unstable at even 60% capacity)[citation needed].

The method described here will allow you to fully utilize your own router and fully bypass your Residential Gateway. It survives reboots, re-authentications, IPv6, and new DHCP leases.

How it Works

Before continuing to the setup, it's important to understand how this method works. This will make configuration and troubleshooting much easier.

Standard Procedure

First, let's talk about what happens in the standard setup (without any bypass). At a high level, the following process happens when the gateway boots up:

  1. All traffic on the ONT is protected with 802.1/X. So in order to talk to anything, the Router Gateway must first perform the authentication procedure. This process uses a unique certificate that is hardcoded on your Residential Gateway.
  2. Once the authentication completes, you'll be able to properly "talk" to the outside. However, all of your traffic will need to be tagged with VLAN ID 0 (a.k.a. VLAN Priority Tagging[1][2]) before the IP gateway will respond.
  3. Once traffic is tagged with VLAN ID 0, your Residential Gateway needs to request a public IPv4 address via DHCP. The MAC address in the DHCP request needs to match that of the MAC address that's assigned to your AT&T account[citation needed]. Other than that, there's nothing special about the DCHPv4 handshake.
  4. After the DHCP lease is issued, the WAN setup is complete. Your LAN traffic is then NAT'd and routed to the outside.

Bypass Procedure

To bypass the Residential Gateway using OPNsense, we can emulate the standard procedure entirely within OPNsense. However, there is a complication; tagging traffic with VLAN ID 0 is not supported through the standard interfaces.

netgraph

This is where netgraph comes in. netgraph allows you to break some rules and build the proper plumbing to make this work. In our case, it simply allows us to tag traffic with VLAN ID 0.

Smart/Managed Switch

Alternatively, you can setup a smart/managed switch between the ONT and OPNsense. Assign two ports to an untagged VLAN, choose an ID other than the default. I chose 100. You can use the rest of the switch ports as usual. I chose to use them as untagged VLAN 1.

D-Link:

image image image

TP-Link:

image image

Now traffic from the ONT should be stripped of its VLAN tag before being sent to OPNsense.

Hopefully, that gives you an idea of what we are trying to accomplish. See the comments and commands in bin/opnatt.sh for details about the netgraph setup.

Setup

Prerequisites

There are some excellent resources on rooting and extracting certificates from various gateways:

Install

  1. Download/clone this repo somewhere you can edit it.

  2. Copy your certificates somewhere they can be accessed by the script, such as the wpa directory.

  3. Edit the configuration variables in bin/opnatt.conf to your setup.

    
# Interface Options

ONT_IF="xx0" EAP_IDENTITY="XX:XX:XX:XX:XX:XX" RG_ETHER="XX:XX:XX:XX:XX:XX"

wpa_supplicant Options

ca_cert="/conf/opnatt/wpa/ca.pem" client_cert="/conf/opnatt/wpa/client.pem" private_key="/conf/opnatt/wpa/private.pem"


4. Copy the repo to `/conf/opnatt`:

```shell
scp -r opnatt root@OPNsense:/conf/
  1. At this point testing the script would be a good idea.
ssh root@OPNsense
/conf/opnatt/bin/opnatt.sh
  1. To start opnatt at boot:
ln /conf/opnatt/bin/opnatt.sh /usr/local/etc/rc.syshook.d/start/94-opnatt

  1. Assign ngeth0 as your WAN interface.

  2. Once again, it's recommended to reboot OPNsense to test that WAN comes up.

  3. Apply your configurations, firewall rules, etc. in OPNsense. Do not add the underlying interface (xx0) to OPNsense.

IPv6 Setup

Big thanks to @pyrodex1980's post on DSLReports for notes on IPv6.

WAN Setup

  1. Go to Interfaces > WAN
  2. Enable IPv6 Configuration Type as DHCP6
  3. Scroll to DCHP6 Client Configuration
  4. Enable Request only an IPv6 prefix
  5. Enable DHCPv6 Prefix Delegation size as 60
  6. Enable Send IPv6 prefix hint
  7. Enable Do not wait for a RA
  8. Save

LAN Setup

  1. Go to Interfaces > LAN
  2. Change the IPv6 Configuration Type to Track Interface
  3. Under Track IPv6 Interface, assign IPv6 Interface to your WAN interface.
  4. Configure IPv6 Prefix ID to 0. You CAN use IPv6 Prefix id 0, as OPNSense does NOT assign a routeable IPv6 address to ngeth0
  5. Save

If you have additional LAN interfaces repeat these steps for each interface except be sure to provide an IPv6 Prefix ID that is not 0 and is unique among the interfaces you've configured so far.

DHCPv6 Server & RA

  1. Go to Services > DHCPv6 Server & RA
  2. Enable DHCPv6 server on interface LAN
  3. Configure a range of ::0001 to ::ffff:ffff:ffff:fffe
  4. Leave Prefix Delegation Range blank.
  5. Configure Prefix Delegation Size to 64
  6. Save
  7. Go to the Router Advertisements tab
  8. Configure Router mode as Stateless DHCP
  9. Save

If you have additional LAN interfaces repeat these steps for each interface.

That's it! Now your clients should be receiving public IPv6 addresses via DHCP6.

Troubleshooting

Logging

Output from opnatt.sh can be found in the OPNsense Web UI under System -> Log Files -> General.

wpa_supplicant

Connect to the wpa_supplicant daemon and view status:

wpa_cli status

tcpdump

Use tcpdump to watch authentication, VLAN and DHCP traffic.

tcpdump -ei $ONT_IF

Verify you are seeing 802.1Q (tagged as vlan0) traffic on your $ONT_IF interface and untagged traffic on ngeth0.

Verify the DHCP request is firing using the MAC address of your Residential Gateway.

If the VLAN0 traffic is being properly handled, next OPNsense will need to request an IP. ngeth0 needs to DHCP using the authorized MAC address. You should see an untagged DCHP request on ngeth0 carry over to the $ONT_IF interface tagged as VLAN0. Then you should get a DHCP response and you're in business.

If you don't see traffic being bridged between ngeth0 and $ONT_IF, then netgraph is not setup correctly.

Promiscuous Mode

opnatt.sh will put $ONT_IF in promiscuous mode via /sbin/ifconfig $ONT_IF promisc. It's unclear if this is required or not in supplicant mode.

Kernel Modules

Confirm the required kernel modules are loaded with kldstat. The following modules are required:

You can edit which kernel modules are loaded by opnatt.sh in opnatt.conf.

netgraph

The netgraph system provides a uniform and modular system for the implementation of kernel objects which perform various networking functions. If you're unfamiliar with netgraph, this tutorial is a great introduction.

Your netgraph should look something like this:

netgraph

In this setup, the igb3 interface is my $ONT_IF.

You can generate your own graphviz via ngctl dot. Copy the output and paste it at webgraphviz.com or render locally with dot.

Try these commands to inspect whether netgraph is configured properly.

  1. Issue ngctl list to list netgraph nodes. Inspect opnatt.sh to verify the netgraph output matches the configuration in the script. It should look similar to this:
Name: igb3 Type: ether ID: 00000004 Num hooks: 1
Name: vlan0 Type: vlan ID: 0000000d Num hooks: 2
Name: ngeth0 Type: eiface ID: 00000010 Num hooks: 1
  1. Inspect the various nodes and hooks:
Local hook Peer name Peer type Peer ID Peer hook
lower vlan0 vlan 0000000d downstream
Local hook Peer name Peer type Peer ID Peer hook
vlan0 ngeth0 eiface 00000010 ether
downstream igb3 ether 00000004 lower
Local hook Peer name Peer type Peer ID Peer hook
ether vlan0 vlan 0000000d vlan0

Reset netgraph

opnatt.sh expects a clean netgraph before it can be ran. To reset a broken netgraph state, try this:

/usr/sbin/ngctl shutdown waneapfilter:
/usr/sbin/ngctl shutdown laneapfilter:
/usr/sbin/ngctl shutdown $ONT_IF:
/usr/sbin/ngctl shutdown $RG_IF:
/usr/sbin/ngctl shutdown o2m:
/usr/sbin/ngctl shutdown vlan0:
/usr/sbin/ngctl shutdown ngeth0:

This will remove netgraph nodes setup by other bypass modes e.g. netgraph bridge. Make sure to define $ON_IF and $RG_IF before running!

Other Methods

pfatt (netgraph w/ bridge)

https://github.com/MonkWho/pfatt

Linux

If you're looking how to do this on a Linux-based router, please refer to this method which utilizes ebtables and some kernel features. The method is well-documented there and I won't try to duplicate it. This method is generally more straight forward than doing this on BSD. However, please submit a PR for any additional notes for running on Linux routers.

VLAN Swap

There is a whole thread on this at DSLreports. The gist of this method is that you connect your ONT, RG and WAN to a switch. Create two VLANs. Assign the ONT and RG to VLAN1 and the WAN to VLAN2. Let the RG authenticate, then change the ONT VLAN to VLAN2. The WAN the DHCPs and your in business.

However, I don't think this works for everyone. I had to explicitly tag my WAN traffic to VLAN0 which wasn't supported on my switch.

FreeBSD

I haven't tried this with native FreeBSD, but I imagine the process is ultimately the same with netgraph. Feel free to submit a PR with notes on your experience.

U-verse TV

See U-VERSE_TV.md

Known Issues

References

Credits

This took a lot of testing and a lot of hours to figure out. If this helped you out, please buy us a coffee.