Assertation failed and segmentation fault when connecting to Check Point VPN

[RGarrido03]

Hi! My university uses a Check Point VPN, whose CLI (snx) is deprecated. I'm trying to use this to adopt a modern & native solution in my pc, however I'm facing an assertation error and a segmentation fault.

All parameters are okay and comply with the values in the Check Point forum, and I'm using the correct certificates (my university's VPN uses TLS certificates for machine authentication, instead of a Pre-Shared Key).

I'm running the VPN through the Network Manager GUI in the GNOME Control Center.

journalctl output is like this:

abr 20 00:15:05 NetworkManager[526]: <info>  [1681946105.3193] audit: op="connection-activate" uuid="04d21ba0-ecfe-452f-9a2b-16e78d5ba28b" name="UA" pid=112811 uid=1000 result="success"
abr 20 00:15:05 nm-l2tp-service[167844]: Check port 1701
abr 20 00:15:05 nm-l2tp-service[167844]: g_file_get_contents: assertion 'filename != NULL' failed
abr 20 00:15:05 kernel: nm-l2tp-service[167844]: segfault at 8 ip 0000555d0f676d82 sp 00007ffcb4923890 error 4 in nm-l2tp-service[555d0f66f000+e000] likely on CPU 4 (core 2, socket 0)
abr 20 00:15:05 kernel: Code: 31 c9 6a 00 48 8b bd 78 fe ff ff 31 d2 48 89 c6 6a 00 ff 15 70 ce 00 00 48 89 85 70 fe ff ff 5a 59 48 85 c0 0f 84 91 09 00 00 <8b> 53 08 48 8b 33 48 89 c7 ff 15 df d0 00 00 85 c0 0f 85 15 07 00
abr 20 00:15:05 systemd[1]: Started Process Core Dump (PID 167851/UID 0).
abr 20 00:15:05 systemd-coredump[167852]: [🡕] Process 167844 (nm-l2tp-service) of user 0 dumped core.

                                          Stack trace of thread 167844:
                                          #0  0x0000555d0f676d82 n/a (nm-l2tp-service + 0xdd82)
                                          #1  0x0000555d0f67c382 n/a (nm-l2tp-service + 0x13382)
                                          #2  0x00007f3f52e0efc7 n/a (libnm.so.0 + 0x109fc7)
                                          #3  0x00007f3f528e4f2e n/a (libffi.so.8 + 0x7f2e)
                                          #4  0x00007f3f528e10bc n/a (libffi.so.8 + 0x40bc)
                                          #5  0x00007f3f528e4383 ffi_call (libffi.so.8 + 0x7383)
                                          #6  0x00007f3f52ae9515 g_cclosure_marshal_generic (libgobject-2.0.so.0 + 0x1a515)
                                          #7  0x00007f3f52d502df n/a (libnm.so.0 + 0x4b2df)
                                          #8  0x00007f3f52ae3210 g_closure_invoke (libgobject-2.0.so.0 + 0x14210)
                                          #9  0x00007f3f52b112f8 n/a (libgobject-2.0.so.0 + 0x422f8)
                                          #10 0x00007f3f52d5a17e n/a (libnm.so.0 + 0x5517e)
                                          #11 0x00007f3f52c4e8ee n/a (libgio-2.0.so.0 + 0x11e8ee)
                                          #12 0x00007f3f52c33288 n/a (libgio-2.0.so.0 + 0x103288)
                                          #13 0x00007f3f529de53b g_main_context_dispatch (libglib-2.0.so.0 + 0x5a53b)
                                          #14 0x00007f3f52a3b219 n/a (libglib-2.0.so.0 + 0xb7219)
                                          #15 0x00007f3f529ddc7f g_main_loop_run (libglib-2.0.so.0 + 0x59c7f)
                                          #16 0x0000555d0f6727a8 n/a (nm-l2tp-service + 0x97a8)
                                          #17 0x00007f3f520cb790 n/a (libc.so.6 + 0x23790)
                                          #18 0x00007f3f520cb84a __libc_start_main (libc.so.6 + 0x2384a)
                                          #19 0x0000555d0f6729b5 n/a (nm-l2tp-service + 0x99b5)

                                          Stack trace of thread 167845:
                                          #0  0x00007f3f521a80dd syscall (libc.so.6 + 0x1000dd)
                                          #1  0x00007f3f52a347b5 g_cond_wait (libglib-2.0.so.0 + 0xb07b5)
                                          #2  0x00007f3f529a8fb4 n/a (libglib-2.0.so.0 + 0x24fb4)
                                          #3  0x00007f3f52a0ff9e n/a (libglib-2.0.so.0 + 0x8bf9e)
                                          #4  0x00007f3f52a0b315 n/a (libglib-2.0.so.0 + 0x87315)
                                          #5  0x00007f3f5212dbb5 n/a (libc.so.6 + 0x85bb5)
                                          #6  0x00007f3f521afd90 n/a (libc.so.6 + 0x107d90)

                                          Stack trace of thread 167847:
                                          #0  0x00007f3f521a29df __poll (libc.so.6 + 0xfa9df)
                                          #1  0x00007f3f52a3b17f n/a (libglib-2.0.so.0 + 0xb717f)
                                          #2  0x00007f3f529ddc7f g_main_loop_run (libglib-2.0.so.0 + 0x59c7f)
                                          #3  0x00007f3f52c3ed5c n/a (libgio-2.0.so.0 + 0x10ed5c)
                                          #4  0x00007f3f52a0b315 n/a (libglib-2.0.so.0 + 0x87315)
                                          #5  0x00007f3f5212dbb5 n/a (libc.so.6 + 0x85bb5)
                                          #6  0x00007f3f521afd90 n/a (libc.so.6 + 0x107d90)

                                          Stack trace of thread 167846:
                                          #0  0x00007f3f521a29df __poll (libc.so.6 + 0xfa9df)
                                          #1  0x00007f3f52a3b17f n/a (libglib-2.0.so.0 + 0xb717f)
                                          #2  0x00007f3f529dd1a2 g_main_context_iteration (libglib-2.0.so.0 + 0x591a2)
                                          #3  0x00007f3f529dd1f2 n/a (libglib-2.0.so.0 + 0x591f2)
                                          #4  0x00007f3f52a0b315 n/a (libglib-2.0.so.0 + 0x87315)
                                          #5  0x00007f3f5212dbb5 n/a (libc.so.6 + 0x85bb5)
                                          #6  0x00007f3f521afd90 n/a (libc.so.6 + 0x107d90)
                                          ELF object binary architecture: AMD x86-64
abr 20 00:15:05 systemd[1]: systemd-coredump@9-167851-0.service: Deactivated successfully.

[dkosovic]

This is where the g_file_get_contents() function is called: https://github.com/nm-l2tp/NetworkManager-l2tp/blob/6c89042b1a17610efb759fa307e0e72eae6802b9/src/nm-l2tp-service.c#L235

For some reason it appears that the ipsec_secrets_file filename that is passed to the g_file_get_contents() function is NULL. The filename is typically /etc/ipsec.secrets (although on Fedora when strongswan is used, it falls back to /etc/strongswan/ipsec.secrets).

Did you build NetworkManager-l2tp from source code? If so, did you override the default /etc/ipsec.secrets filename --with-nm-ipsec-secrets= and the /etc/ipsec.d secrets dir with --with-nm-ipsec-secrets-dir= ?

I've seen bug reports from 2017 that the g_file_get_contents() function in glib was producing a false positive, but wouldn't think and current linux distro would have that bug.

[dkosovic]

Actually, it doesn't make sense that the filename is NULL as it checks earlier in the code that the ipsec_secrets_file file exists with the g_file_test() function before it calls the has_include_ipsec_secrets() which calls the problematic g_file_get_contents() :

https://github.com/nm-l2tp/NetworkManager-l2tp/blob/6c89042b1a17610efb759fa307e0e72eae6802b9/src/nm-l2tp-service.c#L699-L701

I guess SELinux or AppArmor could be interfering with opening the contents of /etc/ipsec.secret and it is producing a misleading error.

Are you able open the contents of /etc/ipsec.secrets without error from the command-line, e.g,:

sudo cat /etc/ipsec.secrets

Otherwise, I think you might have a buggy glib library which has the g_file_get_contents() function.

[dkosovic]

You can try manually adding the following line to the end of /etc/ipsec.secrets and see if it makes a difference:

include ipsec.d/ipsec.nm-l2tp.secrets

As the code you are having issues with is basically checking if that include line is there and adds it if it isn't.

If it's able to get past that section of code, the certificate details for strongswan will then be written to the /etc/ipsec.d/ipsec.nm-l2tp.secrets file.

[RGarrido03]

Thank you for the quick reply, and I'm sorry for the delay, yesterday was a pretty busy day.

This is where the g_file_get_contents() function is called:

https://github.com/nm-l2tp/NetworkManager-l2tp/blob/6c89042b1a17610efb759fa307e0e72eae6802b9/src/nm-l2tp-service.c#L235

For some reason it appears that the ipsec_secrets_file filename that is passed to the g_file_get_contents() function is NULL. The filename is typically /etc/ipsec.secrets (although on Fedora when strongswan is used, it falls back to /etc/strongswan/ipsec.secrets).

Did you build NetworkManager-l2tp from source code? If so, did you override the default /etc/ipsec.secrets filename --with-nm-ipsec-secrets= and the /etc/ipsec.d secrets dir with --with-nm-ipsec-secrets-dir= ?

I've seen bug reports from 2017 that the g_file_get_contents() function in glib was producing a false positive, but wouldn't think and current linux distro would have that bug.

I didn't build NetworkManager-l2tp from the source code, because I got it from the Arch repos. And no, I didn't override any default, because I'm connecting through either the GNOME Shell's quick panel or the GNOME Control Center.

Actually, it doesn't make sense that the filename is NULL as it checks earlier in the code that the ipsec_secrets_file file exists with the g_file_test() function before it calls the has_include_ipsec_secrets() which calls the problematic g_file_get_contents() :

https://github.com/nm-l2tp/NetworkManager-l2tp/blob/6c89042b1a17610efb759fa307e0e72eae6802b9/src/nm-l2tp-service.c#L699-L701

I guess SELinux or AppArmor could be interfering with opening the contents of /etc/ipsec.secret and it is producing a misleading error.

Are you able open the contents of /etc/ipsec.secrets without error from the command-line, e.g,:

sudo cat /etc/ipsec.secrets

Otherwise, I think you might have a buggy glib library which has the g_file_get_contents() function.

I don't have SELinux or AppArmor, and yes, I'm able to cat that file successfully. I really think I have a buggy glib library.

You can try manually adding the following line to the end of /etc/ipsec.secrets and see if it makes a difference:

include ipsec.d/ipsec.nm-l2tp.secrets

As the code you are having issues with is basically checking if that include line is there and adds it if it isn't.

If it's able to get past that section of code, the certificate details for strongswan will then be written to the /etc/ipsec.d/ipsec.nm-l2tp.secrets file.

Unfortunately, this doesn't change anything. But I'm not using strongswan, I'm using libreswan instead:

sudo pacman -Q | grep "swan"
libreswan 4.10-1
networkmanager-libreswan 1.2.16-1

[dkosovic]

I think I was mistaken in the reading the journalctl output for the g_file_get_contents() function. Here are all the location in the source code where that function is called:

https://github.com/nm-l2tp/NetworkManager-l2tp/blob/6c89042b1a17610efb759fa307e0e72eae6802b9/src/nm-l2tp-service.c#L235

https://github.com/nm-l2tp/NetworkManager-l2tp/blob/d3495d52ad3b1b55eba187124f4ad006fae55b95/shared/nm-l2tp-crypto-nss.c#L178

https://github.com/nm-l2tp/NetworkManager-l2tp/blob/d3495d52ad3b1b55eba187124f4ad006fae55b95/shared/nm-l2tp-crypto-openssl.c#L32

https://github.com/nm-l2tp/NetworkManager-l2tp/blob/d3495d52ad3b1b55eba187124f4ad006fae55b95/shared/nm-utils/nm-shared-utils.c#L2079

The first location in src/nm-l2tp-service.c was where I previously thought the issue was, but it is not possible as you can only get there if you are using strongswan. So please undo the changes I suggested to make to /etc/ipsec.secrets file, that include line is not needed for libreswan.

shared/nm-l2tp-crypto-nss.c or shared/nm-l2tp-crypto-openssl.c at first seem unlikely as there are checks for NULL elsewhere in the code, but they are the prime suspects as they are related to processing of the TLS certificate code.

shared/nm-utils/nm-shared-utils.c can be eliminated as the corresponding function that it is located in, isn't used.

I'll have to get back to you after I've done some testing. I did remove some deprecated code that OpenSSL 3.1 was complaining about with the last release of NetworkManager-l2tp, I may have gone too far and removed something I shouldn't have.

[dkosovic]

shared/nm-l2tp-crypto-openssl.c is probably the trouble maker, there are code paths were a NULL value can be set for the filename that is passed to the g_file_get_contents() function. I won't get a chance to work on the code till the weekend.

[RGarrido03]

Thank you for your findings. Take your time, no worries :)

[dkosovic]

Can you provide a screenshot of the IPsec Properties dialog box that shows the TLS certificate machine authentication fields? I want to see what fields have been filled in and what type of certificates and keys are being used to make it easier to reproduce the issue.

I've committed something in the code to no longer cause a core dump if the filename is NULL, it prevents the assertion, but doesn't fix the actual issue. I suspect something the the Gtk4 pull request from last year may have broken something with the certificate handling in the GUI.

[dkosovic]

I haven't been able to reproduce the issue, but suspect it is one of the following filenames that is empty (i.e. (None) or NULL) in the IPsec settings :

• CA certificate filename
• Machine certificate filename
• Machine private key filename

If you are using a PKCS#12 or .p12 certificate file, then all 3 of the above should automatically get filled out to the same filename.

If you are use a PEM or DER certificate file along with a private key, then only the CA certificate filename can optionally be empty.

The NMACertChooser code introduced with commit https://github.com/nm-l2tp/NetworkManager-l2tp/commit/22e06721c4c06fdb841b6f45bb83e304fd8da8fd for Gtk4 support isn't as user friendly as the previous GtkFileChooserButton for handling certificates, unfortunately the GtkFileChooserButton no longer exists with Gtk4

[RGarrido03]

I'm sorry for the delay; I must confess that I got the email yesterday, but I completely forgot to reply 👀

Can you provide a screenshot of the IPsec Properties dialog box that shows the TLS certificate machine authentication fields? I want to see what fields have been filled in and what type of certificates and keys are being used to make it easier to reproduce the issue.

I've committed something in the code to no longer cause a core dump if the filename is NULL, it prevents the assertion, but doesn't fix the actual issue. I suspect something the the Gtk4 pull request from last year may have broken something with the certificate handling in the GUI.

Here it is! Do you want me to build this from source, so that I can test the commit?

I haven't been able to reproduce the issue, but suspect it is one of the following filenames that is empty (i.e. (None) or NULL) in the IPsec settings :

• CA certificate filename
• Machine certificate filename
• Machine private key filename

If you are using a PKCS#12 or .p12 certificate file, then all 3 of the above should automatically get filled out to the same filename.

If you are use a PEM or DER certificate file along with a private key, then only the CA certificate filename can optionally be empty.

The NMACertChooser code introduced with commit 22e0672 for Gtk4 support isn't as user friendly as the previous GtkFileChooserButton for handling certificates, unfortunately the GtkFileChooserButton no longer exists with Gtk4

I'm only using .crt files, not .p12 ones. The machine private key input is disabled in the GUI, so I think it's not needed, right?

[dkosovic]

No worries.

The machine private key field is definitely needed and that's what is causing the 'filename != NULL' assertion to fail.

If the GUI recognises what sort of certificate it is, it makes the machine private key field editable. But looks like it doesn't know what sort of certificate it is.

Could you try the file and openssl x509 commands (only need a little of the output from the latter) on the CShell_Certificate.crt certificate file to see what sort of certificate it is.

$ file rsa_2048.crt
rsa_2048.crt: Certificate, Version=3

$ openssl x509 -in rsa_2048.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            3c:2b:eb:04:86:9b:e2:a8:c7:bc:1e:fa:24:ca:0b:f2:a9:cc:45:e6
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = AU, ST = Queensland, L = Brisbane, OU = Test, CN = TestCN
        Validity
            Not Before: Apr 23 08:35:42 2023 GMT
            Not After : Sep  7 08:35:42 2050 GMT
...

[RGarrido03]

The output is as follows:

file CShell_Certificate.crt                        
CShell_Certificate.crt: PEM certificate

openssl x509 -in CShell_Certificate.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            27:d3:e3:77:54:8b:00:84:05:c0:e9:98:bc:d2:24:4a:77:10:93:aa
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = IL, O = Check Point, OU = Mobile Access, CN = Check Point Mobile
        Validity
            Not Before: Apr 19 21:44:01 2023 GMT
            Not After : Apr 16 21:44:01 2033 GMT
        Subject: C = IL, O = Check Point, OU = Mobile Access, CN = Check Point Mobile

I got it through the vpn.sh script in this repo, after connecting to the Check Point VPN using that. Truth is, though, I need a private machine key, but I don't know where exactly I can get it lmao.

[dkosovic]

.crt certificates are generally DER certificates, but in this case it is PEM, that shouldn't matter.

I think there is a bug with the NMACertChooser from the libnma package you are using. I'm not able to reproduce the private key not enabled issue with a PEM .crt certificate. You could try renaming CShell_Certificate.crt to CShell_Certificate.pem and adding the renamed certificate back and see if it makes a difference.

[RGarrido03]

Just tested; as soon as I added the renamed certificate to the NM GUI, it unlocked the private key field!

I didn't input any certificate there because, as I said, I don't know where to get it. Because of that, it still fails with that assertation error. Should I input the same CShell_Certificate.pem in the Machine private key field? I'll test it.

[RGarrido03]

Well, when I try that it fails with the following warning: <warn> [1682360308.0184] vpn[0x55f5bf51c410,04d21ba0-ecfe-452f-9a2b-16e78d5ba28b,"UA"]: failed to connect: 'Error decrypting private key file '/home/ruben/Downloads/CShell_Certificate.pem', so I guess I can't just use the same certificate for both public and private keys. I'll try to grab it somewhere in the system, idk.

[RGarrido03]

I just found out that there is also a .p12 certificate in the same folder (that repo's config) as the .crt one. I tried it, and it automatically filled the 3 fields as you said. However, once I try to connect to the VPN, it now asks for a Machine Certificate Password. What the heck is this?

[dkosovic]

A PKCS#12 (or .p12) file stores a private key and its corresponding certificate, optionally it may also contain the corresponding CA certificate. That's why all 3 machine auth fields show the same .p12 file when a .p12 file is used.

A private key is definitely required for the L2TP/IPsec client machine authentication, be it in a standalone key file or in a .p12 file. I suspect the standalone key file is there somewhere, but not sure what the file extension would be. A private key is just a text file which begins and ends with the following:

-----BEGIN PRIVATE KEY-----

-----END PRIVATE KEY-----

or if the private key is encrypted with a password (aka passphrase), it begins and ends with:

-----BEGIN ENCRYPTED PRIVATE KEY-----

-----END ENCRYPTED PRIVATE KEY----

So you could grep "PRIVATE KEY" * in the folder(s) to see if there is a private key file somewhere, hopefully an unencrypted one especially if you don't know the corresponding password or passphrase.

A PKCS#12 (or .p12) file often needs a passphrase (aka password) to unlock it, similarly to private key that is encrypted with a password or passphrase.

NetworkManager-l2tp will only prompt you for the Machine Certificate Password if it was determined the .p12 file needs a passphrase (aka password). The IPsec settings also has a password field where the password can be stored in the machine authentication fields if you don't want to be prompted for the Machine Certificate Password.

The following openssl command will most likely prompt for the passphrase before it will show the info for the .p12 file :

openssl pkcs12 -info -in CShell_Certificate.p12

[dkosovic]

I don't understand how the vpn.sh script in that repo works. It seems to fetch the TLS server certificate from a localhost website. I don't understand how you'll then be able use that for TLS client machine's certificate. I'm not sure where the .p12 file came from, but you normally specify the passphrase (aka password) at certificate creation time.

On the following Check Point page for L2TP/IPsec, they talk about client machine's certificate : https://sc1.checkpoint.com/documents/R81/WebAdminGuides/EN/CP_R81_RemoteAccessVPN_AdminGuide/Topics-VPNRG/L2TP-Clients.htm

The L2TP VPN server validates the client machine's certificate and the L2TP client does a validation of the VPN server certificate.

[RGarrido03]

Unfortunately, I can't find the private key, i.e., any file containing

-----BEGIN ENCRYPTED PRIVATE KEY-----

-----END ENCRYPTED PRIVATE KEY----

It is indeed very strange on how the Check Point VPN works. The .p12 file is in the cshell folder, I also don't know how that file came from. I mean, the vpn.sh script doesn't even mention this type of file...

But yeah, (un)fortunately the problem is not related to NetworkManager-l2tp. I have to make some deeper research; it would be awesome to connect to the VPN through the GNOME Shell. Thank you so much for the time you spent on this issue!!

[dkosovic]

When I added support for certificates in NetworkManager-l2tp back in 2019, I used Windows Server 2019 in a VM as the L2TP test server. I hadn't used TLS certificates in a while, so it was all fuzzy in memory.

I just noticed I still had the script I used to generate the machine certificates. Turns out I used the ipsec pki command from strongswan to generate machine certificates as it was easier than using openssl. But still had to use openssl for generating the PKCS#12 container file.

The following has good info on what is required for a machine certificate and also how to make them compatible with Windows, macOS, iOS and Android clients:

• https://docs.strongswan.org/docs/5.9/interop/windowsCertRequirements.html
• https://docs.strongswan.org/docs/5.9/pki/pkiQuickstart.html

The simple PSK is way easier.

I'll close this issue and wish you luck.