freedomofpress / dangerzone

Take potentially dangerous PDFs, office documents, or images and convert them to safe PDFs
https://dangerzone.rocks/
GNU Affero General Public License v3.0
3.62k stars 172 forks source link

Migrate to Qt6 before Qt5 end-of-life #211

Open deeplow opened 2 years ago

deeplow commented 2 years ago

Mirroring https://github.com/freedomofpress/securedrop-client/issues/1562:

According to Qt, Qt 5.15 will reach EOL on 2023-05-26 for non-subscribers. We should ensure we migrate to a supported version before then.

We have a PR that already does this change but it is blocked on distros lacking Pyside6 (Qt6) packages.

deeplow commented 1 year ago

I checked on Dec. 12th 2022 the current availability of the python3-pyside6 package which we need for Qt6 in python. I checked these for the currently supported systems. It is not looking promising:

Distro version Qt6 PySide6
Ubuntu 22.10 (kinetic) :heavy_check_mark: :x:
Ubuntu 22.04 (jammy) :heavy_check_mark: :x:
Ubuntu 20.04 (focal) :x: :x:
Debian 12 (bookworm) :heavy_check_mark: :x:
Debian 11 (bullseye) :heavy_check_mark: :x:
Fedora 37 :heavy_check_mark: :x:
Fedora 36 :heavy_check_mark: :x:
Fedora 35 :heavy_check_mark: :x:

Another good website for checking this is repology (compare, for example, with PySide6).

eloquence commented 1 year ago

For 0.4.1 we're aiming to make headway towards Qt6 support on Mac and Windows by direct bundling, with further testing and research for Linux distributions. Moving Linux distributions to Qt6 for 0.4.1 would be a stretch goal.

deeplow commented 1 year ago

PySide6 won't be available for Fedora distributions any time soon. So we need to find a way to bundle it into our .rpm ideally before it reaches EOL.

Distributing Dangerzone RPMs with bundled PySide6

Option A: bundling with bdist_rpm (will be deprecated)

Note: In the process of looking into this issue, I found out that bdist_rpm is on its way to being deprecated (https://github.com/freedomofpress/dangerzone/issues/298).

bdist_rpm is the build tool we're using for creating .rpm files.

  1. we generate python wheels for the missing packages.

    pip install wheel
    mkdir wheels
    pip wheel --wheel-dir ./wheels PySide6

    This will download the wheels for the missing packages and all of their dependencies. All we have to do after is packaging them into the RPM and install these wheels at install time.

  2. Then we bundle them with bdist_rpm. There may be two ways to do this:

    • Option A) (A post-install script that installs the python wheels](https://stackoverflow.com/questions/33687249/python-bdist-rpm-not-using-install-requires) (I haven't confirmed that this works)
    • Option B) by modifying the .spec file generation. We need to add to the file extra extra lines to include %files for wheels built in (1.) and perhaps a %py3_install_wheel.

      To achieve this, we can override the bdist_rpm class and wrap its _make_spec_file() method (docs about extending commands). The final setup.py would look something like this:

      import distutils.command.bdist_rpm as orig
      from distutils.core import setup
      
      class bdist_rpm(orig.bdist_rpm):
          """Specialized Python source builder."""
      
          def _make_spec_file():
                  spec = orig.bdist_rpm._make_spec_file(self)
                  # modify spec file here
      setup(
         ...
         cmdclass={'bdist_rpm': bdist_rpm},
         ...
      )

      Note: We should ensure that the installation of the PySide wheels are conditional (to ensure we don't overwrite package-manager-maintained ones) or that it somehow is installed in a virtual environment that only Dangerzone can use.

Option B: Freezing

Bundling everything in our code may be another viable solution. We already do this for MacOS via CxFreeze, which also supports building RPMs with its relatively recent inclusion in its source code of the soon-to-be-deprecated bdist_rpm.

Related docs: https://docs.python-guide.org/shipping/freezing/#freezing-your-code-ref

Option C: our own fedora packaging

Instead of having the packaging files for fedora generated automatically from a setup.py file, we can make them ourselves. https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/

We can use as the base the spec file generated by setuptools and keep developing it ourselves. It can be generated with python setup.py bdist_rpm --spec-only.

deeplow commented 1 year ago

Not sure this is fixed @apyrgio. Linux systems still run on it.

apyrgio commented 1 year ago

Oh, good catch, this should have been a "Refs".

apyrgio commented 1 year ago

As of 2023-06-13, I don't see any known distro, besides Alpine and Arch Linux that package PySide6. Also, the Qt5 support has officially ended in May 2023. Thankfully, MacOS and Windows builds use PySide6, but we immediately need to update our Debian/Fedora packages as well.

Regarding @deeplow's comment above, I'd be more in favor of option C, as it doesn't affect the package creation of Dangerzone, and because we can always update the package irrespective of the Dangerzone releases. For reference, this is the way Arch Linux packages PySide6: https://gitlab.archlinux.org/archlinux/packaging/packages/pyside6/-/blob/main/PKGBUILD

eloquence commented 1 year ago

We had the same discussion on the SecureDrop Client side of things and decided that given that Debian is continuing to provide security updates, the migration timeline is not as critical as upstream EOL suggests, and we can wait until the Bookworm switchover: https://github.com/freedomofpress/securedrop-client/issues/1562#issuecomment-1464185785

That said, I don't know if Fedora applies security updates in a similar manner after a project has reached EOL upstream (@legoktm, tagging in case you have insights on that). And of course there are other reasons to make everything consistent.

legoktm commented 1 year ago

That said, I don't know if Fedora applies security updates in a similar manner after a project has reached EOL upstream

Usually they do, but it can take longer, especially for the older supported release (currently F37). Looking at https://packages.fedoraproject.org/pkgs/qt5-qtbase/qt5-qtbase/fedora-38-updates.html and https://packages.fedoraproject.org/pkgs/qt5-qtbase/qt5-qtbase/fedora-37-updates.html the maintainer was applying security fixes from a month ago but that was also before the EOL, so it's unclear if they'll continue. (Also skimming those issues, none of them would have affected Dangerzone nor SDW AFAICT.)

deeplow commented 11 months ago

This has started biting us back in Dangerzone development since PySide2 stopped being supported on Fedora 39.

Option C: our own fedora packaging

Instead of having the packaging files for fedora generated automatically from a setup.py file, we can make them ourselves. https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/

We can use as the base the spec file generated by setuptools and keep developing it ourselves. It can be generated with python setup.py bdist_rpm --spec-only.

Since we already tackled this part we may now consider want to consider the possibility of shipping PySide6 with our own package. Either that or us maintaining the PySide6 package on fedora.

deeplow commented 10 months ago

Option C: our own fedora packaging

SecureDrop stumbled upon a similar issue: https://github.com/freedomofpress/securedrop/pull/6884. There a venv was used to ship newer versions of software that is unsupported by distros.

apyrgio commented 10 months ago

It seems that shipping PySide6 is the best way forward, so I did a bit of research on what that means, what are the alternatives, and how we can actually do it.

Interaction with Qt System Libraries

First thing I checked and want to address. I was afraid that providing a PySide6 package might break, if a Python binding points to a system Qt library that gets updated in the meantime. Thankfully, this is not the case. The PyPI version of PySide6 provides a Qt/ directory that surprisingly holds all the Qt libraries. The Python modules point to these libraries, and these libraries do not point to any Qt system libraries.

This is further explained in https://doc.qt.io/qtforpython-6/quickstart.html:

Having Qt installed in your system will not interfere with your PySide6 installation if you do it via pip install, because the Python packages (wheels) include already Qt binaries. Most notably, style plugins from the system won’t have any effect on PySide applications.

I have tested this by removing any Qt system library from my dev environment, and just had PySide6 installed. Spinning up the Dangerzone GUI works! At the end of the day, I guess this is expected, since Dangerzone works on Windows/MacOS as well, which do not have any Qt system libraries.

Note that in our development environment, we install a GUI-related Qt6 library (libqt6gui6). I thought this was a requirement, but turns out we only use it to bring in the rest of the GUI libraries (xkb, fonts, opengl). Of course, we will need to depend on these libraries directly, if we want to package PySide6.

Code Verification

Ok, so we want to package PySide6. Let's assume that the Python wheel is the best candidate, since it contains the Qt libraries as well. Where do we download it from? How can we verify its contents?

So, we don't seem to have a way to verify Qt sources or binary packages via signatures (ideally), but we can use hashes provided by the maintainer and HTTPS (less ideal).

On the other hand, that's kind of the current situation on Windows/MacOS. We rely on a TOFU model of trust, in broad strokes. That is, we assume that the file hashes we grab from PyPI and store in our Poetry lock file are correct the moment we update our Poetry lock file, and malicious interference can only take place and get detected afterwards.

Ways to distribute PySide6

There are various ways we can bring PySide6 into the user's environment. I've tried to evaluate those:

  1. Vendor PySide6 into the Dangerzone RPM package. Basically, this would require shipping /usr/lib/python3.11/site-packages/{PySide6,shiboken6} within Dangerzone itself. It's a viable alternative, but this means that any global build flags for the Dangerzone RPM would apply to the PySide6 parts, which is not so simple as we will see next.

  2. Create a virtualenv, install PySide6 in it, and ship it. SecureDrop used dh-virtualenv in the past successfully. In Fedora land, I see the following related projects:

    I haven't experimented with them yet mostly due to time constraints, and because I wanted to have the option to exercise a bit more control on the contents on the virtualenv (e.g., not just blindly install the package from PyPI, without hash verification).

  3. Package PySide6 as a separate RPM package. Let's see some tools that can help us in this job:

    • https://github.com/fedora-python/pyp2rpm: Throws the following error:

      Unable to extract package metadata from .whl archive. This might be caused by an old .whl format version. You may ask the upstream to upload fresh wheels created with wheel >= 0.17.0 or to upload an sdist as well to workaround this problem.

    • https://github.com/openSUSE/py2pack: Probably does not work with binary-only wheels, and throws the following error:

      KeyError: 'urls'

    • https://github.com/jordansissel/fpm: Does not work with binary-only wheels (see this open issue), and throws the following error:

      fpm: ERROR: Could not find a version that satisfies the requirement PySide6 (from versions: none)

It seems that packaging PySide6 as a separate RPM gives us most control over the process, so we're going to evaluate this option first.

Concerns

1. What will happen if Fedora actually packages PySide6 after our package is out?

This depends on which version they package:

In both cases, if another system package has PySide6 as a dependency and does not get the exact same version, this may lead to issues. It's good to make sure that we always work with the latest PySide6 version in any case.

2. How to build the package in a way that we install the necessary PySide6 requirements in the user's system (xkb, fonts, opengl)?

My guess is we have to specify them manually. Probably best to figure out which are missing in a clean Fedora container.

3. The resulting package size is big, roughly 150MiB compressed. Can we make it smaller?

My guess is that this is probably due to the bundled Qt libraries, since the same PySide2 package is ~7 MiB. We could probably remove the libraries that we are not using, but that could break any other program that attempts to import PySide6 (see 1).

4. PySide6's shared libraries are built with the manylinux project. Are they safe to be installed in the user's system?

I've already seen that automatic dependency detection goes haywire once rpmbuild runs ldd on the shared libraries. E.g.:

libstdc++.so.6()(64bit)                                    
libstdc++.so.6(CXXABI_1.3)(64bit)                          
libstdc++.so.6(CXXABI_1.3.5)(64bit)                        
libstdc++.so.6(CXXABI_1.3.9)(64bit)                        
libstdc++.so.6(GLIBCXX_3.4)(64bit)                         
libstdc++.so.6(GLIBCXX_3.4.11)(64bit)                      
libstdc++.so.6(GLIBCXX_3.4.14)(64bit)                      
libstdc++.so.6(GLIBCXX_3.4.15)(64bit)
libstdc++.so.6(GLIBCXX_3.4.17)(64bit)
libstdc++.so.6(GLIBCXX_3.4.18)(64bit)
libstdc++.so.6(GLIBCXX_3.4.19)(64bit)
libstdc++.so.6(GLIBCXX_3.4.20)(64bit)
libstdc++.so.6(GLIBCXX_3.4.21)(64bit)
libstdc++.so.6(GLIBCXX_3.4.22)(64bit)
libstdc++.so.6(GLIBCXX_3.4.9)(64bit)

This probably affects tools that do some sort of analysis on these libraries, but Dangerzone does not seem to be affected.

5. Is it worth building the package on our own, based on OpenSUSE's specfile?

I guess that this will lead to a smaller package (because we will depend on the system Qt6 libraries), but this will resurface the concern regarding the divergence of the bindings and the system libraries, which I cannot answer yet confidently.

apyrgio commented 10 months ago

tl;dr: Building PySide6 from source requires digging into their build files. It's not worth the hassle though, since PySide6 relies on private Qt6 APIs that can change at any moment, and can easily break the user's system in case of Qt6 updates.

Building PySide6 from Source

We have established that we can package PySide6 from a PyPI package, but this comes with some drawbacks:

So, it would be worth checking if building PySide6 from source will help alleviate these problems.

I've used Fedora 39 as a build platform, and tried to follow the official instructions in Qt's site. I also consulted OpenSUSE's python-pyside6 specfile at times.

After hunting down missing packages and fixing some CMake problems, I'm currently stuck at the following error:

[ 11%] Building C object sources/pyside6/qtexampleicons/CMakeFiles/QtExampleIcons.dir/module.c.o
/pyside-setup/sources/pyside6/qtexampleicons/module.c:4:10: fatal error: Python.h: No such file or directory
    4 | #include <Python.h>
      |          ^~~~~~~~~~
compilation terminated.

I believe this error is CMake-related because:

  1. The Python.h header file exists in /usr/include/python3.12/Python.h
  2. CMake detects that it should include headers from /usr/include/python3.12:

    -- PYTHON_INCLUDE_DIRS:    /usr/include/python3.12
    -- SHIBOKEN_PYTHON_INCLUDE_DIRS computed to value: '/usr/include/python3.12' 
  3. OpenSUSE has a patch that overrides the detected paths for Python libraries, which indicates a similar build problem.

I could dig up further, but at this point I've allocated too much time on this simple task.

Examining the (would-be) Final RPM

While we haven't managed to build PySide6 from source and package it as an RPM, we can download the RPM from OpenSUSE's site, and examine it.

PySide6 RPM requirements ``` # rpm -qp --requires /python3-pyside6-6.6.0-1.2.x86_64.rpm warning: /python3-pyside6-6.6.0-1.2.x86_64.rpm: Header V3 RSA/SHA512 Signature, key ID 29b700a4: NOKEY /sbin/ldconfig /sbin/ldconfig libQt63DAnimation.so.6()(64bit) libQt63DAnimation.so.6(Qt_6)(64bit) libQt63DCore.so.6()(64bit) libQt63DCore.so.6(Qt_6)(64bit) libQt63DExtras.so.6()(64bit) libQt63DExtras.so.6(Qt_6)(64bit) libQt63DInput.so.6()(64bit) libQt63DInput.so.6(Qt_6)(64bit) libQt63DLogic.so.6()(64bit) libQt63DLogic.so.6(Qt_6)(64bit) libQt63DRender.so.6()(64bit) libQt63DRender.so.6(Qt_6)(64bit) libQt63DRender.so.6(Qt_6.6.0_PRIVATE_API)(64bit) libQt6Bluetooth.so.6()(64bit) libQt6Bluetooth.so.6(Qt_6)(64bit) libQt6Charts.so.6()(64bit) libQt6Charts.so.6(Qt_6)(64bit) libQt6Charts.so.6(Qt_6.6.0_PRIVATE_API)(64bit) libQt6Core.so.6()(64bit) libQt6Core.so.6(Qt_6)(64bit) libQt6Core.so.6(Qt_6.6)(64bit) libQt6Core.so.6(Qt_6.6.0_PRIVATE_API)(64bit) libQt6DBus.so.6()(64bit) libQt6DBus.so.6(Qt_6)(64bit) libQt6DataVisualization.so.6()(64bit) libQt6DataVisualization.so.6(Qt_6)(64bit) libQt6Designer.so.6()(64bit) libQt6Designer.so.6(Qt_6)(64bit) libQt6Graphs.so.6()(64bit) libQt6Graphs.so.6(Qt_6)(64bit) libQt6Gui.so.6()(64bit) libQt6Gui.so.6(Qt_6)(64bit) libQt6Help.so.6()(64bit) libQt6Help.so.6(Qt_6)(64bit) libQt6HttpServer.so.6()(64bit) libQt6HttpServer.so.6(Qt_6)(64bit) libQt6Location.so.6()(64bit) libQt6Location.so.6(Qt_6)(64bit) libQt6Multimedia.so.6()(64bit) libQt6Multimedia.so.6(Qt_6)(64bit) libQt6MultimediaWidgets.so.6()(64bit) libQt6MultimediaWidgets.so.6(Qt_6)(64bit) libQt6Network.so.6()(64bit) libQt6Network.so.6(Qt_6)(64bit) libQt6Network.so.6(Qt_6.6.0_PRIVATE_API)(64bit) libQt6NetworkAuth.so.6()(64bit) libQt6NetworkAuth.so.6(Qt_6)(64bit) libQt6Nfc.so.6()(64bit) libQt6Nfc.so.6(Qt_6)(64bit) libQt6OpenGL.so.6()(64bit) libQt6OpenGL.so.6(Qt_6)(64bit) libQt6OpenGLWidgets.so.6()(64bit) libQt6OpenGLWidgets.so.6(Qt_6)(64bit) libQt6Pdf.so.6()(64bit) libQt6Pdf.so.6(Qt_6)(64bit) libQt6PdfWidgets.so.6()(64bit) libQt6PdfWidgets.so.6(Qt_6)(64bit) libQt6Positioning.so.6()(64bit) libQt6Positioning.so.6(Qt_6)(64bit) libQt6PrintSupport.so.6()(64bit) libQt6PrintSupport.so.6(Qt_6)(64bit) libQt6Qml.so.6()(64bit) libQt6Qml.so.6(Qt_6)(64bit) libQt6Qml.so.6(Qt_6.6.0_PRIVATE_API)(64bit) libQt6Quick.so.6()(64bit) libQt6Quick.so.6(Qt_6)(64bit) libQt6Quick3D.so.6()(64bit) libQt6Quick3D.so.6(Qt_6)(64bit) libQt6QuickControls2.so.6()(64bit) libQt6QuickControls2.so.6(Qt_6)(64bit) libQt6QuickWidgets.so.6()(64bit) libQt6QuickWidgets.so.6(Qt_6)(64bit) libQt6RemoteObjects.so.6()(64bit) libQt6RemoteObjects.so.6(Qt_6)(64bit) libQt6Scxml.so.6()(64bit) libQt6Scxml.so.6(Qt_6)(64bit) libQt6Sensors.so.6()(64bit) libQt6Sensors.so.6(Qt_6)(64bit) libQt6SerialBus.so.6()(64bit) libQt6SerialBus.so.6(Qt_6)(64bit) libQt6SerialPort.so.6()(64bit) libQt6SerialPort.so.6(Qt_6)(64bit) libQt6SpatialAudio.so.6()(64bit) libQt6SpatialAudio.so.6(Qt_6)(64bit) libQt6Sql.so.6()(64bit) libQt6Sql.so.6(Qt_6)(64bit) libQt6StateMachine.so.6()(64bit) libQt6StateMachine.so.6(Qt_6)(64bit) libQt6Svg.so.6()(64bit) libQt6Svg.so.6(Qt_6)(64bit) libQt6SvgWidgets.so.6()(64bit) libQt6SvgWidgets.so.6(Qt_6)(64bit) libQt6Test.so.6()(64bit) libQt6Test.so.6(Qt_6)(64bit) libQt6TextToSpeech.so.6()(64bit) libQt6TextToSpeech.so.6(Qt_6)(64bit) libQt6UiTools.so.6()(64bit) libQt6UiTools.so.6(Qt_6)(64bit) libQt6WebChannel.so.6()(64bit) libQt6WebChannel.so.6(Qt_6)(64bit) libQt6WebEngineCore.so.6()(64bit) libQt6WebEngineCore.so.6(Qt_6)(64bit) libQt6WebEngineQuick.so.6()(64bit) libQt6WebEngineQuick.so.6(Qt_6)(64bit) libQt6WebEngineWidgets.so.6()(64bit) libQt6WebEngineWidgets.so.6(Qt_6)(64bit) libQt6WebSockets.so.6()(64bit) libQt6WebSockets.so.6(Qt_6)(64bit) libQt6Widgets.so.6()(64bit) libQt6Widgets.so.6(Qt_6)(64bit) libQt6Xml.so.6()(64bit) libQt6Xml.so.6(Qt_6)(64bit) libc.so.6()(64bit) libc.so.6(GLIBC_2.14)(64bit) libc.so.6(GLIBC_2.2.5)(64bit) libc.so.6(GLIBC_2.32)(64bit) libc.so.6(GLIBC_2.34)(64bit) libgcc_s.so.1()(64bit) libgcc_s.so.1(GCC_3.0)(64bit) libm.so.6()(64bit) libm.so.6(GLIBC_2.2.5)(64bit) libm.so.6(GLIBC_2.35)(64bit) libm.so.6(GLIBC_2.38)(64bit) libpyside6.abi3.so.6.6()(64bit) libpyside6qml.abi3.so.6.6()(64bit) libpython3.11.so.1.0()(64bit) libshiboken6.abi3.so.6.6()(64bit) libstdc++.so.6()(64bit) libstdc++.so.6(CXXABI_1.3)(64bit) libstdc++.so.6(CXXABI_1.3.9)(64bit) libstdc++.so.6(GLIBCXX_3.4)(64bit) libstdc++.so.6(GLIBCXX_3.4.14)(64bit) libstdc++.so.6(GLIBCXX_3.4.21)(64bit) python(abi) = 3.11 rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(PayloadIsZstd) <= 5.4.18-1 ```

I'd like to draw your attention to these requirements:

[...]
libQt6Core.so.6()(64bit)
libQt6Core.so.6(Qt_6)(64bit)
libQt6Core.so.6(Qt_6.6)(64bit)
libQt6Core.so.6(Qt_6.6.0_PRIVATE_API)(64bit)
[...]

What do these requirements tell us? That the PySide6 package requires a Qt6 library that is compatible with the Qt 6.6 API, as well as the private Qt 6.6 API. This introduces two problems:

  1. If the system Qt library gets upgraded to a version > 6.6, then our PySide6 package will conflict with the new library, until we push out a new package. In the meantime, the user cannot update their system.
    • This is not theoretical. Since Fedora 37's release, qt6-qtbase has been upgraded from 6.3.1 -> 6.4.1 -> 6.4.2 (1 minor upgrade and 1 patch upgrade)
  2. Even if we make our PySide6 RPM to broadly require Qt6 libraries, assuming that Qt folks won't break API compatibility for minor and patch releases, the fact that private APIs are used is much more severe.
    • Fedora maintainers used to rebuild all of their Qt5 packages for this reason (see email thread). They stopped doing that only when Qt5 entered maintenance mode. Their caution was not unwarranted, since Telegram Desktop, for example, would break even in the case of patch releases (see email reply).

Bottom line is: offering a PySide6 package that relies on system Qt6 libraries is bound to break, and cause system updates to stall. Fixing it requires us to be alert of such changes and publish a new PySide6 package whenever that happens.

deeplow commented 10 months ago

Bottom line is: offering a PySide6 package that relies on system Qt6 libraries is bound to break, and cause system updates to stall. Fixing it requires us to be alert of such changes and publish a new PySide6 package whenever that happens.

Doesn't sound sustainable at all.

apyrgio commented 7 months ago

It seems that Debian has backported some patches from PySide6 in their own PySide2 package, in order to make it work with Python 3.12. We have to wait and see if Fedora picks up on those patches and reinstates PySide2.

We haven't tested rebuilding PySide2 in Fedora 39 with these patches, because they appeared after our packaging effort. We don't know if these patches can be cleanly applied to Fedora's PySide2 source, nor what happens if Qt5 gets updated after we publish a patched PySide2 (probably nothing because PySide2 development is stalled).

In any case, we can revisit this solution if our wheel approach is less maintainable, and maybe offer it upstream as well.

apyrgio commented 5 months ago

Quick update, Fedora Rawhide now provides python3-pyside6 6.7.0. We expect this to appear in the rest of the stable versions soon.

apyrgio commented 2 months ago

Another important update: Fedora 40 now provides python3-pyside6 6.7.2 through its updates channel: https://packages.fedoraproject.org/pkgs/python-pyside6/python3-pyside6/fedora-40-updates.html 🎉

The only version that does not have python3-pyside6 is Fedora 39, which will be EOL on November 12th, 2024