opnsense / core

OPNsense GUI, API and systems backend
https://opnsense.org/
BSD 2-Clause "Simplified" License
3.28k stars 727 forks source link

interfaces: DHCPv6 incoming requirements for random server origin port #7527

Closed antipiot closed 3 months ago

antipiot commented 3 months ago

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

Describe the bug

My opnsense installation does not seems to use / apply the prefix delegated by my internet service provider router

To Reproduce

Setup IPV6 with DHCP on the router side: i have a/48 delegated On Opnsense, enable DHCPv6 - / Send IPv6 prefix hint / Prefix delegation size /57 (or whatever does the same) / Request only an IPv6 prefix Check what is the delegated network using else: cat ifctl -6 -l -p or the system - route - status - and look for loopback addresses that match your ipv6 ip range.

Expected behavior

A network with the delegated subnet is applied / used by opnsense The command and routes status show's and address that looks like 1:1:1:1000/52

Describe alternatives you considered

Tested on 2 similar installations (same provider / different bare metla hardware for opnsense) with opnsense v23 and v24 with the same behavior

Relevant log files

I've wiresharked the thing and it looks like the delagation is correctly sendt to opnsense: image

Other informations I've already deployed IPv6 with success with another internet provider.

Environment

opnsense bare metal 24.1.8

Thanks for any hint!

Regards

fichtner commented 3 months ago

Hi @antipiot,

I see your point, but no configuration or system output we can trace this by. There's also the ability to set debug log level for dhcp6c but for some reason I expect this to be configuration related.

Cheers, Franco

antipiot commented 3 months ago

Hi @fichtner !

Thanks for your help. For sure it can be configuration related, but... where...

If i'm not mistaken the only needed configuration to get the delegation is on the interface settings. Thoses are my settings. image

I've indeed played around here to try to find a sweet spot without luck. Is there anything elses i can try?

I will try to enable debug log level for DHCPv6

antipiot commented 3 months ago

In the logs for my interface i have this message:

2024-06-14T08:44:33 | Notice | dhcp6c | dhcp6c_script: on igc1 ignored

As far as i can tell there should be a $reason in this message? image

Does not seems i have thoses messages on my another setup.

fichtner commented 3 months ago

An empty $REASON? Looks like a good thing to ignore ;)

Your capture says Prefix length: 52, why do you try to set 55?

antipiot commented 3 months ago

@fichtner When requesting a 55 i have a /52 delegated, already tried to set to 52 without success. May i ask you how to enable verbose DHCPv6 logs? couldnt find the right way to do it

antipiot commented 3 months ago

i do have the feeling it's ISP related, but how to prove / debug that

If i just set my WAN interface to DHCP, not requesting delegation, i do get an ipv6 on the WAN interface and i can ping ipv6 addresses on the internet with success.

After some more testings it looks like the Slaac part work, not the DHCP part as i dont have an adress in the DHCP range i was supposed to get.

fichtner commented 3 months ago

All of this rounds rather strange.

It's best to enable DHCPv6 debugging and reboot and then look for all the dhcp6c lines in System: Log Files: General to avoid speculation.

antipiot commented 3 months ago

All of this rounds rather strange.

It's best to enable DHCPv6 debugging and reboot and then look for all the dhcp6c lines in System: Log Files: General to avoid speculation.

How to enable DHCPv6 debugging?

edit: found in Interfaces->Settings->IPv6 DHCP Log Level.

antipiot commented 3 months ago

@fichtner here's a part of what i have in logs

dhcp.log

fichtner commented 3 months ago

That snippet already shows that nobody answers to the SOLICIT of dhcp6c.

antipiot commented 3 months ago

That snippet already shows that nobody answers to the SOLICIT of dhcp6c.

Yes. does not looks like opnsense related.

antipiot commented 3 months ago

That snippet already shows that nobody answers to the SOLICIT of dhcp6c.

But why is there a Advertise answer showing up there?

image

On a setup where everything is working as expected we can indeed se the right transactions: image

This tends to prove that Opnsense is not getting the answer for some fishy reasons if i'm not mistaken.

fichtner commented 3 months ago

I have no idea where you capture or which machine replies. Relevant things are edited out, logs are partial and so far not a single ifconfig output to help get this some structure.

antipiot commented 3 months ago

I have no idea where you capture or which machine replies. Relevant things are edited out, logs are partial and so far not a single ifconfig output to help get this some structure.

Thanks for your time @fichtner, really appreciate.

This was captured using the "capture" function of opnsense. filtering on the implied WAN interface and IPV6

I'm sorry but you can perfectly identify the interfaces in thoses captures. The "sollicit" message from interface "f3" is issued by Opnsense, the Advertise from "90" is the ISP Router. The second cature is from another installation and was looking to show a working DHCPv6 trade

Logs are partiel because i was trying not to submerge with not relevent informations :) The fact that the request was "timeouting" sounded sufficient to me, but if you telle me that you need the full log, i can surely provide it to you!

antipiot commented 3 months ago

@fichtner Ok. i think i'm starting to understand Packets were blocked by firewall because they weren't send by port 546 WAN |   | 2024-06-18T08:49:41 | [fe80:::79f3]:48457 | [fe80::c790]:546 | udp | Default deny / state violation rule

The router does not send the packet from the expected port

IPv6 UDP | fe80::/10 | 546 | fe80::/10 | 546 | | |   | allow dhcpv6 client in WAN

I added a rule to allow this but i still get no confirmation.

Is it possible that Opnsense wont use a dhcp message that is issued by another port than 546?

Regards

fichtner commented 3 months ago

@antipiot https://en.wikipedia.org/wiki/DHCPv6#Port_numbers

Clients listen for DHCP messages on UDP port 546. Servers and relay agents listen for DHCP messages on UDP port 547.

If your ISP behaviour differs you need manual rules for sure.

Cheers, Franco

fichtner commented 3 months ago

Then again, maybe we can loosen that requirement, but it's quite unusual.

antipiot commented 3 months ago

@antipiot https://en.wikipedia.org/wiki/DHCPv6#Port_numbers

Clients listen for DHCP messages on UDP port 546. Servers and relay agents listen for DHCP messages on UDP port 547.

If your ISP behaviour differs you need manual rules for sure.

Cheers, Franco

Yes, i do think that aswell. I'm trying to find the RFC for this, that specify that the DHCP server has to answer using port 546

I've tried to add a manual rule but it does not looks to be enough: can i try something else?

In your link the provided example does not match the opnsense configuration aswell: image

fichtner commented 3 months ago

You can edit the following, just remove the from_port specification

https://github.com/opnsense/core/blob/34cafe3e9835cb48c41ad063d2aba2700e7f701a/src/etc/inc/filter.lib.inc#L352

It's really odd this pins the port from 546 to 546, because you either have a server to client 547 -> 546 or client to server 546 -> 547.

antipiot commented 3 months ago

You can edit the following, just remove the from_port specification

https://github.com/opnsense/core/blob/34cafe3e9835cb48c41ad063d2aba2700e7f701a/src/etc/inc/filter.lib.inc#L352

It's really odd this pins the port from 546 to 546, because you either have a server to client 547 -> 546 or client to server 546 -> 547.

Do you find, by any chances, the RFC that request the messages to be sent using either port 546 or 547? couldnt find this on the RFC 8415 -> https://datatracker.ietf.org/doc/html/rfc8415#page-24 image

That one specify the messages needs to be received on thoses ports as they are "listen" requirements

fichtner commented 3 months ago

I know the BPF filter in dhclient (IPv4) enforces the strict sending port, but the dhcp6c client may or may not.

The origin of this rule is https://github.com/pfsense/pfsense/commit/dbcddabcdf7e which gives no reference to what it tried to achieve. It has never been changed since.

fichtner commented 3 months ago

It would possibly make sense to condense this back into a single rule and relax the incoming server port. I'm not sure if everything is done over link-local but I kind of expect it is.

fichtner commented 3 months ago

(historically speaking that IPv6 stuff was probably just copied over from IPv4 filtering requirements)

antipiot commented 3 months ago

It would possibly make sense to condense this back into a single rule and relax the incoming server port. I'm not sure if everything is done over link-local but I kind of expect it is.

It was from all my testings at least.

antipiot commented 3 months ago

You can edit the following, just remove the from_port specification

https://github.com/opnsense/core/blob/34cafe3e9835cb48c41ad063d2aba2700e7f701a/src/etc/inc/filter.lib.inc#L352

It's really odd this pins the port from 546 to 546, because you either have a server to client 547 -> 546 or client to server 546 -> 547.

This edit does not seems to be sufficient, i'm still getting a "sollicit / advertise" loop.

fichtner commented 3 months ago

capture from the WAN port would help then... it's doing something right ;)

fichtner commented 3 months ago

This is what I have in mind, but I can't test before 24.1.9 is out in a bit... https://github.com/opnsense/core/commit/a3a5e7832

antipiot commented 3 months ago

capture from the WAN port would help then... it's doing something right ;)

Rebooted everything aaaand -> image Hourrah

fichtner commented 3 months ago

Thanks for debugging. I'll test the patch, ask you to try it too (with a clean reboot) and then we will likely add it for 24.7 :)

antipiot commented 3 months ago

Thanks for debugging. I'll test the patch, ask you to try it too (with a clean reboot) and then we will likely add it for 24.7 :)

Will do with pleasure. Thanks for the fix.

fichtner commented 3 months ago

Ok patch works fine on my end... to try from your end make sure you are on 23.1.9 (just came out) and apply the patch cleanly with the following commands:

# opnsense-revert opnsense && opnsense-patch a3a5e78323

Cheers, Franco

antipiot commented 3 months ago

Ok patch works fine on my end... to try from your end make sure you are on 23.1.9 (just came out) and apply the patch cleanly with the following commands:

# opnsense-revert opnsense && opnsense-patch a3a5e78323

Cheers, Franco

Will do! DId you thinked about adjusting the automatic firewall rule aswell?

@fichtner Ok. i think i'm starting to understand Packets were blocked by firewall because they weren't send by port 546 WAN | | 2024-06-18T08:49:41 | [fe80:::79f3]:48457 | [fe80::c790]:546 | udp | Default deny / state violation rule

The router does not send the packet from the expected port

IPv6 UDP | fe80::/10 | 546 | fe80::/10 | 546 | | | | allow dhcpv6 client in WAN

I added a rule to allow this but i still get no confirmation.

Is it possible that Opnsense wont use a dhcp message that is issued by another port than 546?

Regards

fichtner commented 3 months ago

DId you thinked about adjusting the automatic firewall rule aswell?

Wasn't that what we did with the patch? Not sure which other automatic rule you could mean.

antipiot commented 3 months ago

DId you thinked about adjusting the automatic firewall rule aswell?

Wasn't that what we did with the patch? Not sure which other automatic rule you could mean

Sorry i was mixing things up!

Patched and had to restart all services - Looks OK