How to add portaudio as a dependency to pyaudio?

[garaiza-93]

In a project I want to package, it requires pyaudio. However, pyaudio depends on portaudio.

Based on the flake template provided in the readme, this is my current flake:

{
  inputs.dream2nix.url = "github:nix-community/dream2nix";
  outputs = inp:
    inp.dream2nix.lib.makeFlakeOutputs {
      systemsFromFile = ./nix_systems;
      config.projectRoot = ./.;
      source = ./.;
      projects = ./projects.toml;

      inject = {
        pyaudio."0.12.2" = [
          [ "portaudio" "19.7.0" ]
        ];
      };

      sourceOverrides = oldSources: {
        pyaudio."0.12.2" = builtins.fetchTarball { url = "https://files.pythonhosted.org/packages/3e/2f/67dc1505002ed6a4ef3f5941d6f07209d4c013a0271dd8d18314f54f6bf0/PyAudio-0.2.12.tar.gz"; sha256 = "1mr1rx84bkryrcbfl6a55ac85fxv6asjyzrjgcjvgnh348qhhs1z"; };
        portaudio."19.7.0" = builtins.fetchTarball { url = "https://github.com/PortAudio/portaudio/archive/refs/tags/v19.7.0.tar.gz"; sha256 = "1mr1rx84bkryrcbfl6a55ac85fxv6asjyzrjgcjvgnh348qhhs1z"; };
      };
    };
}

and my projects.toml file:

# projects.toml file describing inputs for dream2nix
#
# To re-generate this file, run:
#   nix run .#detect-projects $source
# ... where `$source` points to the source of your project.
#
# If the local flake is unavailable, alternatively execute the app from the
# upstream dream2nix flake:
#   nix run github:nix-community/dream2nix#detect-projects $source

[main]
name = "main"
relPath = ""
subsystem = "python"
translator = "pip"
translators = [ "pip",]

[main.subsystemInfo]
pythonAttr = "python3"

Log from building this flake.:

Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/3lzamq2gv9nlv9day08rcjg>
source root is qr2r4wgccfxf40fk2i7inrxsihhrrcv3-source
setting SOURCE_DATE_EPOCH to timestamp 315619200 of file qr>
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
building
cp: -r not specified; omitting directory '/nix/store/1xwigs>
lines 1-19/19 (END)...skipping...
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/3lzamq2gv9nlv9day08rcjg1dcp6hlyr-qr2r4wgccfxf40fk2i7inrxsihhrrcv3-source/
source root is qr2r4wgccfxf40fk2i7inrxsihhrrcv3-source
setting SOURCE_DATE_EPOCH to timestamp 315619200 of file qr2r4wgccfxf40fk2i7inrxsihhrrcv3-source/windows.png
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
building
cp: -r not specified; omitting directory '/nix/store/1xwigs68nflwmwmxswsmva5qgrslqz39-source'

This results in the flake failing to build. A different approach I have tried is to instead use the portaudio package from nixos-unstable in extraSetupDeps as described in the documentation Flake with attempted workaround:

{
  inputs.dream2nix.url = "github:nix-community/dream2nix";
  inputs.nixpkgs.url = "github:nix-community/nixpkgs/nixos-unstable";
  outputs = inp:
    inp.dream2nix.lib.makeFlakeOutputs {
      systemsFromFile = ./nix_systems;
      config.projectRoot = ./.;
      source = ./.;
      projects = ./projects.toml;

      extraSetupDeps = [
        inp.nixpkgs.pkgs.portaudio
      ];
    };
}

...which then fails again, with the message flake 'github:nix-community/nixpkgs/e2b4abe3c8f2e09adfc6a52007841d1b96c89371' has an unsupported attribute 'edition', at /nix/store/k57k9avjpkzpclfi02sglsqhig7skwzn-source/flake.nix:4:3

So i'm stumped. How would I go about adding portaudio so that the pyaudio wheel can build?

[garaiza-93]

I've made some changes, but pyaudio still fails to build; same result.

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; 
    dream2nix.url = "github:nix-community/dream2nix";
  };

  outputs = { self, dream2nix, nixpkgs }:
  let
    l = nixpkgs.lib // builtins;

    allPkgs =
      l.map
      (system: nixpkgs.legacyPackages.${system})
      ["x86_64-linux" "aarch64-linux"];
  in
    dream2nix.lib.makeFlakeOutputs rec {
      systemsFromFile = ./nix_systems;
      config.projectRoot = ./.;
      source = ./.;
      projects = ./projects.toml;
      pkgs = allPkgs;

      packageOverrides = {
        pyaudio = {
          buildInputs = old: old ++ [ pkgs.portaudio ];
        };
      };
    };
}

[garaiza-93]

I have also tried using a flake that someone kindly wrote for me:

{
  inputs = {
    dream2nix.url = "github:nix-community/dream2nix";
  };

  outputs =
    { self
    , dream2nix
    } @ inp:
    let
      pkgs = inp.dream2nix.inputs.nixpkgs.legacyPackages.x86_64-linux;
    in
    dream2nix.lib.makeFlakeOutputs {
      systems = [ "x86_64-linux" ];
      config.projectRoot = ./.;
      source = ./.;
      projects = ./projects.toml;
      packageOverrides = {
        main.add-portaudio = {
          overrideAttrs = ol: {
            buildInputs = (ol.buildInputs or [ ]) ++
              [
                pkgs.portaudio
                pkgs.libxkbcommon
                pkgs.fontconfig
                pkgs.freetype
                pkgs.wayland
                pkgs.xorg.libXcomposite
              ];
          };
        };
      };
    };
}

Good news: pyaudio builds Bad news: auto-patchelf fails to patch many, many missing libraries.

error: auto-patchelf could not satisfy dependency libxcb-icccm.so.4 wanted by /nix/store/s1z66kivplmv65i58vxxs4502hl2vwlg-python3.10-main/lib/python3.10/site-packages/PyQt5/Qt5/plugins/platforms/libqxcb.so
error: auto-patchelf could not satisfy dependency libxcb-image.so.0 wanted by /nix/store/s1z66kivplmv65i58vxxs4502hl2vwlg-python3.10-main/lib/python3.10/site-packages/PyQt5/Qt5/plugins/platforms/libqxcb.so
error: auto-patchelf could not satisfy dependency libxcb-util.so.1 wanted by /nix/store/s1z66kivplmv65i58vxxs4502hl2vwlg-python3.10-main/lib/python3.10/site-packages/PyQt5/Qt5/plugins/platforms/libqxcb.so
error: auto-patchelf could not satisfy dependency libxcb-keysyms.so.1 wanted by /nix/store/s1z66kivplmv65i58vxxs4502hl2vwlg-python3.10-main/lib/python3.10/site-packages/PyQt5/Qt5/plugins/platforms/libqxcb.so
error: auto-patchelf could not satisfy dependency libxcb-render-util.so.0 wanted by /nix/store/s1z66kivplmv65i58vxxs4502hl2vwlg-python3.10-main/lib/python3.10/site-packages/PyQt5/Qt5/plugins/platforms/libqxcb.so
error: auto-patchelf could not satisfy dependency libQt5EglFSDeviceIntegration.so.5 wanted by /nix/store/s1z66kivplmv65i58vxxs4502hl2vwlg-python3.10-main/lib/python3.10/site-packages/PyQt5/Qt5/plugins/platforms/libqeglfs.so
error: auto-patchelf could not satisfy dependency libdrm.so.2 wanted by /nix/store/s1z66kivplmv65i58vxxs4502hl2vwlg-python3.10-main/lib/python3.10/site-packages/PyQt5/Qt5/plugins/platforms/libqlinuxfb.so

and so much more.

I have attempted to add more build inputs as shown in this flake, but it's silly how many I have to do.

[garaiza-93]

I have figured this out, and hopefully no one else has to bang their head on the wall for hours like I did. I solved the last bit by adding another override to use pkgs.python310Packages.pyqt5, such that the working flake is:

{
  inputs.dream2nix.url = "github:nix-community/dream2nix";
  outputs = inp:
    let
      pkgs = inp.dream2nix.inputs.nixpkgs.legacyPackages.x86_64-linux;
    in
    inp.dream2nix.lib.makeFlakeOutputs {
      systems = [ "x86_64-linux" ];
      config.projectRoot = ./.;
      source = ./.;
      projects = ./projects.toml;
      packageOverrides.main = {
        pyaudio.overrideAttrs = oldAttrs: {
          buildInputs = old: old ++ [
            pkgs.portaudio
          ];
        };
        pyqt5.overrideAttrs = oldAttrs: {
          buildInputs = old: old ++ [
            pkgs.python310Packages.pyqt5
          ];
        };
      };
    };
}

However, could the last bit with auto-patchelf be due to something dream2nix related, or was it something with the package from pip? I will close the issue once I get an answer to this question.

[InLaw]

because I cannot find any working example with any translator perhaps someone could help (with this one)?

only trying to reproduce those code parts with

- system: `"x86_64-linux"` 
 - host os: `Linux 5.15.88, NixOS, 22.11 (Raccoon), 22.11.1705.b83e7f5a04a` 
 - multi-user?: `yes` 
 - sandbox: `yes` 
 - version: `nix-env (Nix) 2.11.1` 
 - channels(root): `"nixos-22.11, nixos-hardware"` 
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

results in:

nix flake show
path:/home/pyaudio?lastModified=1674244907&narHash=sha256-99rgQgCDNKBl35bMQK0SP7SQZJp9y033vAU3acRq7Hg=
├───apps
│   └───x86_64-linux
│       └───detect-projects: app
└───packages
    └───x86_64-linux
        ├───main: package 'main'
        └───resolveImpure: package 'resolve'

direnv allow
direnv: loading ~/pyaudio/.envrc
direnv: using flake
error: flake 'path:/home/pyaudio' does not provide attribute 'devShells.x86_64-linux.default', 'devShell.x86_64-linux', 'packages.x86_64-linux.default' or 'defaultPackage.x86_64-linux'