Python environment with `matplotlib` and `jupyter` causes run-time Qt version conflicts in LXQt desktop environment

[ShamrockLee]

Describe the bug

When constructing a python environment wrapper, picking up matplotlib and jupyter, executing python3 inside a terminal emulator on LXQt desktop environment, and plotting with plot() provided by matplotlib.pyplot, MatPlotLib complains

Cannot mix incompatible Qt library (5.15.11) with this library (5.15.12)
Aborted (core dumped)

Steps To Reproduce

Steps to reproduce the behavior:

1. Boot into a NixOS on commit cf28ee2 nixos-23.11 branch, where the version of Qt 5 is 5.15.11.
2. Start a graphical session with LXQt desktop environment.
3. Open a terminal emulator (I use Alacritty).
4. Type lxqt-about --version and see the Qt version. Mine is
5. cd to the Nixpkgs project directory.
6. Check out nixos-unstable or nixos-23.11 (56528ee or later) where the Qt 5 version is 5.15.12 or later.
7. See the Qt 5 version with nix eval .#qt5.full.name. The output on my machine is "qt-full-5.15.12".
8. Enter the Python interactive environment by running nix run --impure --expr "(import ./. { }).python3.withPackages (ps: with ps; [ matplotlib jupyter ])".
9. Type import matplotlib.pyplot as plt and press enter
10. Type plt.plot([1, 2, 3], [1, 1, 1]) and press enter

One-liner of the Python execution part: nix shell --impure --expr "(import ./. { }).python3.withPackages (ps: with ps; [ matplotlib jupyter ])" -c python3 -c "__import__('matplotlib.pyplot').pyplot.plot([1, 2, 3], [1, 1, 1])" (It cannot be replaced with nix run due to a Nix bug NixOS/nix#8900.)

Expected behavior

There should be no error and no abortion. The interactive environment should print something like [<matplotlib.lines.Line2D object at 0x7fb179f10d50>] after plot(), and show the plot after plt.show(). The one-liner should print nothing.

Screenshots

Additional context

$ lxqt-about -v
lxqt-about 1.4.0
liblxqt   1.4.0
Qt        5.15.11

Notify maintainers

@lovek323 @veprbl (MatPlotLib maintainers) @costrouc (author of a Jupyter refactoring commit 6102dd1991e0cd5d3fa0e04ac6934bb90c0ea438) @FRidh (Python package committer) @romildo (LXQt maintainer)

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

---

Add a :+1: reaction to issues you find important.

[phiadaarr]

I believe I have a very related problem.

My set-up is:

flake.nix

{
  description = "Reproducer";

  inputs = {
    nixpkgs.url = "nixpkgs/nixos-24.05";
    flake-utils.url = "github:numtide/flake-utils";
  };
  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; };
        myPyPkgs = pkgs.python3Packages;

        base-dependencies = with myPyPkgs; [ matplotlib numpy ];
        bad-jupyter = with myPyPkgs; [ jupyter ];
        good-jupyter = with myPyPkgs; [ pkgs.jupyter ];

      in {
        devShells.default = pkgs.mkShell {
          nativeBuildInputs = base-dependencies;
        };

        devShells."badJupyter" = pkgs.mkShell {
          nativeBuildInputs = base-dependencies ++ bad-jupyter;
        };

        devShells."goodJupyter" = pkgs.mkShell {
          nativeBuildInputs = base-dependencies ++ good-jupyter;
        };

      });
}

flake.lock

{
  "nodes": {
    "flake-utils": {
      "inputs": {
        "systems": "systems"
      },
      "locked": {
        "lastModified": 1710146030,
        "narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=",
        "owner": "numtide",
        "repo": "flake-utils",
        "rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a",
        "type": "github"
      },
      "original": {
        "owner": "numtide",
        "repo": "flake-utils",
        "type": "github"
      }
    },
    "nixpkgs": {
      "locked": {
        "lastModified": 1722221733,
        "narHash": "sha256-sga9SrrPb+pQJxG1ttJfMPheZvDOxApFfwXCFO0H9xw=",
        "owner": "NixOS",
        "repo": "nixpkgs",
        "rev": "12bf09802d77264e441f48e25459c10c93eada2e",
        "type": "github"
      },
      "original": {
        "id": "nixpkgs",
        "ref": "nixos-24.05",
        "type": "indirect"
      }
    },
    "root": {
      "inputs": {
        "flake-utils": "flake-utils",
        "nixpkgs": "nixpkgs"
      }
    },
    "systems": {
      "locked": {
        "lastModified": 1681028828,
        "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
        "owner": "nix-systems",
        "repo": "default",
        "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
        "type": "github"
      },
      "original": {
        "owner": "nix-systems",
        "repo": "default",
        "type": "github"
      }
    }
  },
  "root": "root",
  "version": 7
}

main.py

import matplotlib.pyplot as plt
import numpy as np

arr = np.ones((12, 12))
plt.imshow(arr)
plt.savefig("foobar.png")

When I run

nix develop .
python3 main.py

all is good.

When I run

nix develop .#badJupyter
python3 main.py

I get:

qt.qpa.plugin: Could not find the Qt platform plugin "xcb" in ""
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

When I run

nix develop .#goodJupyter
python3 main.py

all is good again.

@lovek323 @veprbl, do you have any insights on this? Apparently python3Packages.jupyter is without maintainers at the moment.

Currently python3Packages.jupyter just packages https://pypi.org/project/jupyter/ from PyPI which installs all kind of things including a qtconsole which is probably incompatible with other Nix components.

What shall we do? Does it make sense to remove python3Packages.jupyter and just point to the jupyter package?

[veprbl]

@phiadaarr Do you really need python3Packages.jupyter? python3Packages.notebook should cover most of use cases.

[veprbl]

The issue in the original post I experience myself. And moreover, there seems to be a requirement to have the same version of wayland stuff in flake as in the host system. And none of this is matplotlib-specific, I suspect.

[phiadaarr]

@phiadaarr Do you really need python3Packages.jupyter? python3Packages.notebook should cover most of use cases.

Thanks for this (potential) workaround (I have not tried it yet). I believe in any way, the behaviour that I describe above is a bug.

The issue in the original post I experience myself. And moreover, there seems to be a requirement to have the same version of wayland stuff in flake as in the host system. And none of this is matplotlib-specific, I suspect.

I am on X11. So it appears to be independent of X11 vs wayland.

[veprbl]

I am on X11. So it appears to be independent of X11 vs wayland.

My information may be outdated, but your issue is a known one, and is Qt-specific. The Qt in nixpkgs is setup to use QT_PLUGIN_PATH environment variable set by wrapQtAppsHook. In that approach, no qt backend can work from a normal "python3" invocation.