VPN client segfaults all the time on Ubuntu

[evert]

Describe the bug Segfault. VPN connection stays intact, but client frontend dies.

VPN version: 2.12.0-kinetic-1

Affected Platforms: Ubuntu 22.10

Additional notes:

SegvAnalysis:
 Segfault happened at: 0x7f266eee7938 <_ZNK15QAbstractSocket7isValidEv+8>:  mov    0x208(%rax),%rax
 PC (0x7f266eee7938) ok
 source "0x208(%rax)" (0x00610274) not located in a known VMA region (needed readable region)!
 destination "%rax" ok
SegvReason: reading unknown VMA
SourcePackage: mozillavpn
Stacktrace:
 #0  0x00007f266eee7938 in QAbstractSocket::isValid() const () from /lib/x86_64-linux-gnu/libQt6Network.so.6
 No symbol table info available.
 #1  0x00007f266eef1b81 in QUdpSocket::hasPendingDatagrams() const () from /lib/x86_64-linux-gnu/libQt6Network.so.6
 No symbol table info available.
 #2  0x000055e9a145504d in ?? ()
 No symbol table info available.
 #3  0x00007f266e16ed65 in ?? () from /lib/x86_64-linux-gnu/libQt6Core.so.6
 No symbol table info available.
 #4  0x00007f266eee7127 in ?? () from /lib/x86_64-linux-gnu/libQt6Network.so.6
 No symbol table info available.
 #5  0x00007f266eeef2b9 in ?? () from /lib/x86_64-linux-gnu/libQt6Network.so.6
 No symbol table info available.
 #6  0x00007f266ff7fa55 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /lib/x86_64-linux-gnu/libQt6Widgets.so.6
 No symbol table info available.
 #7  0x00007f266e11a0c8 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /lib/x86_64-linux-gnu/libQt6Core.so.6
 No symbol table info available.
 #8  0x00007f266e36ebef in ?? () from /lib/x86_64-linux-gnu/libQt6Core.so.6
 No symbol table info available.
 #9  0x00007f26705f14f9 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
 No symbol table info available.
 #10 0x00007f2670646228 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
 No symbol table info available.
 #11 0x00007f26705eecb0 in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
 No symbol table info available.
 #12 0x00007f266e36dfa0 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /lib/x86_64-linux-gnu/libQt6Core.so.6
 No symbol table info available.
 #13 0x00007f266e12668b in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /lib/x86_64-linux-gnu/libQt6Core.so.6
 No symbol table info available.
 #14 0x00007f266e1217fc in QCoreApplication::exec() () from /lib/x86_64-linux-gnu/libQt6Core.so.6
 No symbol table info available.
 #15 0x000055e9a1445591 in ?? ()
 No symbol table info available.
 #16 0x000055e9a14462e5 in ?? ()
 No symbol table info available.
 #17 0x000055e9a1433e43 in ?? ()
 No symbol table info available.
 #18 0x000055e9a13d23f6 in ?? ()
 No symbol table info available.
 #19 0x00007f266d223510 in __libc_start_call_main (main=main@entry=0x55e9a13d23b0, argc=argc@entry=1, argv=argv@entry=0x7ffc5c9fdf08) at ../sysdeps/nptl/libc_start_call_main.h:58
         self = <optimized out>
         result = <optimized out>
         unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140721862467336, 1789064429723940109, 0, 140721862467352, 94461936267312, 139803073220640, -1788859646757735155, -1830481411082980083}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x7ffc5c9fdf08}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
         not_first_call = <optimized out>
 #20 0x00007f266d2235c9 in __libc_start_main_impl (main=0x55e9a13d23b0, argc=1, argv=0x7ffc5c9fdf08, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc5c9fdef8) at ../csu/libc-start.c:381
 No locals.
 #21 0x000055e9a13dd605 in ?? ()
 No symbol table info available.
StacktraceAddressSignature: /usr/bin/mozillavpn:11:/usr/lib/x86_64-linux-gnu/libQt6Network.so.6.3.1+6e938:/usr/lib/x86_64-linux-gnu/libQt6Network.so.6.3.1+78b81:/usr/bin/mozillavpn+be04d:/usr/lib/x86_64-linux-gnu/libQt6Core.so.6.3.1+dbd65:/usr/lib/x86_64-linux-gnu/libQt6Network.so.6.3.1+6e127:/usr/lib/x86_64-linux-gnu/libQt6Network.so.6.3.1+762b9:/usr/lib/x86_64-linux-gnu/libQt6Widgets.so.6.3.1+1aa55:/usr/lib/x86_64-linux-gnu/libQt6Core.so.6.3.1+870c8:/usr/lib/x86_64-linux-gnu/libQt6Core.so.6.3.1+2dbbef:/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7400.0+394f9:/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7400.0+8e228:/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7400.0+36cb0:/usr/lib/x86_64-linux-gnu/libQt6Core.so.6.3.1+2dafa0:/usr/lib/x86_64-linux-gnu/libQt6Core.so.6.3.1+9368b:/usr/lib/x86_64-linux-gnu/libQt6Core.so.6.3.1+8e7fc
StacktraceTop:
 QAbstractSocket::isValid() const () from /lib/x86_64-linux-gnu/libQt6Network.so.6
 QUdpSocket::hasPendingDatagrams() const () from /lib/x86_64-linux-gnu/libQt6Network.so.6
 ?? ()
 ?? () from /lib/x86_64-linux-gnu/libQt6Core.so.6
 ?? () from /lib/x86_64-linux-gnu/libQt6Network.so.6

┆Issue is synchronized with this Jira Bug

[data-sync-user]

➤ Andrea Marchesini commented:

Owen Kirby thoughts about this crash?

[oskirby]

This looks to be the same crash as reported in #5049 VPN-3384 and seems to be something related to the handling of the DNS ping sender. I haven't really had much luck narrowing down the problem any further than that so far.

However, in order to trigger the crash I needed to put my device through a suspend/resume cycle and it would crash on wakeup. Furthermore it would only manifest on real hardware (worked fine in a VM).

[karldw]

Just a quick note from the other issue -- I can trigger this by just having wifi disconnect, without going through suspend/resume.

[bakulf]

@karldw can you write a STR? I would like to reproduce the issue. Tell me this is correct:

1. VPN is on
2. disconnect the wifi (or disable it?) Thanks!

[karldw]

That's right.

More detailed version:

1. VPN is on, user is logged in, wifi is connected
2. Something interrupts the wifi connection. Any of these work for me: suspend, disable wifi on computer, disconnect from wifi (but leave enabled), or externally interrupt wifi network.
3. Allow wifi to reconnect
4. VPN crashes shortly after wifi reconnects

[data-sync-user]

➤ Valentina Virlics commented:

We tracked this here - https://mozilla-hub.atlassian.net/browse/VPN-3488 ( https://mozilla-hub.atlassian.net/browse/VPN-3488|smart-link ) - I think are identical.

I’ve re-checked this, using last 2.13 build from release branch, on my Kinetic VM.

Attaching video and logs.

Note: it seems that the tunnel is still on.

!kinetic-crash.gif|width=1481,height=866!

[^mozillavpn-2023-1-17.txt]

Andrea Marchesini Let me know if you need something else.

[data-sync-user]

➤ Andrea Marchesini commented:

Santiago Andrigo we should uplift the fix to 2.13. Are you OK with it?

[data-sync-user]

➤ Santiago Andrigo commented:

If the issue is indeed as described, the VPN client would be completely broken for Linux and it would be completely warranted to uplift. If we do, can you please regenerate the RC for Linux so that QA can use it for regression?Valentina Virlics please holler if you disagree.

[evert]

As a user, it's mostly broken but not completely ;)

[data-sync-user]

➤ Andrea Marchesini commented:

I think we should include it in 2.13. Also because other platforms could crash similarly.

[data-sync-user]

➤ Valentina Virlics commented:

Andrea Marchesini Santiago Andrigo We did not start the regression on Linux yes, so I would say we are ok.

In other words, please let us know when it is uplifted and what is the Linux build ID from PPA.

Thank you!

[data-sync-user]

➤ Andrea Marchesini commented:

I merged the fix into 2.13. it should be already available on the PPA

[data-sync-user]

➤ Santiago Andrigo commented:

Thank you both!

[data-sync-user]

➤ Valentina Virlics commented:

Andrea Marchesini For some reason, while installing from PPA still brings the build from 01/19. Any idea why?

[data-sync-user]

➤ Valentina Virlics commented:

Seems like the PPA got updated!

Verified this as fixed on the last 2.13.0 (2.13.0~rc20230123004344-kinetic1) build on Linux Kinetic.

The Mozilla VPN app no longer crashes after reconnecting the internet.

[data-sync-user]

➤ Andrea Marchesini commented:

Owen Kirby can check what is going on here? Thanks!

[data-sync-user]

➤ Valentina Virlics commented:

Andrea Marchesini this bug IS fixed. Do we need to check something else?

[data-sync-user]

➤ Andrea Marchesini commented:

no. all good! thanks.