v12.1: EAP-TLS using a PFPKI certificate doesn't work on EL8

[nqb]

Describe the bug https://gitlab.com/inverse-inc/packetfence/-/jobs/3322328462

This is what we have in packetfence.log:

Nov 15 16:49:15 pfel8dev httpd.aaa-docker-wrapper[3603]: httpd.aaa(7) INFO: [mac:00:03:00:11:11:01] handling radius autz request: from switch_ip => (172.18.200.201), connection_type => Ethernet-EAP,switch_mac => (44:38:39:00:00:12), mac
=> [00:03:00:11:11:01], port => 8, username => "ARRAY(0x559b041d0dd8)" (pf::radius::authorize)
Nov 15 16:49:15 pfel8dev httpd.aaa-docker-wrapper[3603]: httpd.aaa(7) INFO: [mac:00:03:00:11:11:01] Instantiate profile catch_dot1x_wired_eap_tls (pf::Connection::ProfileFactory::_from_profile)
Nov 15 16:49:15 pfel8dev httpd.aaa-docker-wrapper[3603]: httpd.aaa(7) INFO: [mac:00:03:00:11:11:01] Found authentication source(s) : 'eaptls' for realm 'null' (pf::config::util::filter_authentication_sources)
Nov 15 16:49:15 pfel8dev httpd.aaa-docker-wrapper[3603]: httpd.aaa(7) INFO: [mac:00:03:00:11:11:01] Using sources eaptls for matching (pf::authentication::match2)
Nov 15 16:49:15 pfel8dev httpd.aaa-docker-wrapper[3603]: httpd.aaa(7) ERROR: [mac:00:03:00:11:11:01] radius authorize failed with error: Recursive data structure detected at this point in the structure: 'username'. Cannot flatten recursi
ve structures. at /usr/local/pf/lib_perl/lib/perl5/Hash/Flatten.pm line 226.

This test is not failing on Debian 11.

=> username => "ARRAY(0x559b041d0dd8)" is not expected

To Reproduce Steps to reproduce the behavior:

1. Try to authenticate using a EAP-TLS certificate against an EAPTLS source

Expected behavior Device registered.

Additional context The certificate used to authenticate looks OK.

[julsemaan]

The issue seems to start inside FreeRADIUS, there are 2 certificates being parsed which leads to double attributes

In FreeRADIUS:

(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls: (TLS) Creating attributes from client certificate
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Serial := "05"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Expiration := "241205132330Z"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Valid-Since := "221116132330Z"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Subject := "/C=CA/ST=Quebec/L=Montreal/O=Inverse/OU=PacketFence/CN=InverseCA10"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Issuer := "/C=CA/ST=Quebec/L=Montreal/O=Inverse/OU=PacketFence/CN=InverseCA10"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Common-Name := "InverseCA10"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Subject-Alt-Name-Email := "mailhog@example.lan"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-X509v3-Basic-Constraints += "CA:TRUE"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-X509v3-Subject-Key-Identifier += "3F:1F:43:C7:CC:C2:55:5A:40:07:A4:91:77:CE:69:05:A7:BD:CC:70"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-X509v3-Authority-Key-Identifier += "keyid:3F:1F:43:C7:CC:C2:55:5A:40:07:A4:91:77:CE:69:05:A7:BD:CC:70\n"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls: (TLS) Creating attributes from client certificate
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Serial := "03"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Expiration := "241205132409Z"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Valid-Since := "221116132409Z"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Subject := "/C=CA/ST=Quebec/L=Montreal/O=Inverse/OU=PacketFence/CN=InverseCA10_user_cert"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Issuer := "/C=CA/ST=Quebec/L=Montreal/O=Inverse/OU=PacketFence/CN=InverseCA10"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Common-Name := "InverseCA10_user_cert"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-Subject-Alt-Name-Email := "mailhog@example.lan"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-X509v3-Extended-Key-Usage += "TLS Web Client Authentication"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-X509v3-Subject-Key-Identifier += "AC:CC:4B:3C:98:38:97:C3:9F:B3:17:EF:3E:52:6D:81:30:D8:AD:12"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-X509v3-Authority-Key-Identifier += "keyid:3F:1F:43:C7:CC:C2:55:5A:40:07:A4:91:77:CE:69:05:A7:BD:CC:70\n"
(25) Wed Nov 16 13:27:17 2022: Debug: eap_tls:   TLS-Client-Cert-X509v3-Extended-Key-Usage-OID += "1.3.6.1.5.5.7.3.2"

Which leads to multiple attributes with TLS-Client-Cert-Common-Name and that's where the username is extracted from

Nov 16 14:44:41 pfel8dev httpd.aaa-docker-wrapper[21738]:           'TLS-Client-Cert-Common-Name' => [
Nov 16 14:44:41 pfel8dev httpd.aaa-docker-wrapper[21738]:                                              'InverseCA10',
Nov 16 14:44:41 pfel8dev httpd.aaa-docker-wrapper[21738]:                                              'InverseCA10_user_cert'
Nov 16 14:44:41 pfel8dev httpd.aaa-docker-wrapper[21738]:                                            ],

Given it works on Debian 11 and that radiusd-auth isn't containerized in Classic PF, the FreeRADIUS build could differ between EL and Debian and cause this issue.

@fdurand, are we able to determine if the FreeRADIUS build on EL is the exact same as what is packaged on Debian 11?

[julsemaan]

Using the radiusd-auth docker container that is Debian based exhibits this exact same behavior so it may not be specific to EL8 itself

[julsemaan]

One solution I would have is to filter out the duplicates in httpd.aaa by relying on the fact the real user cert has TLS-Client-Cert-X509v3-Extended-Key-Usage += "TLS Web Client Authentication" but it seems there is no way to correlate which username has that attribute. Perhaps we could in unlang in FreeRADIUS

[nqb]

Issue has been fixed after @fdurand rebuild FreeRADIUS package.

Feel free to re-open @julsemaan if I missed something.