nix-community / poetry2nix

Convert poetry projects to nix automagically [maintainer=@adisbladis,@cpcloud]
MIT License
887 stars 451 forks source link

overrides for package torchvision is missing native build dependencies. #1589

Open panda2134 opened 7 months ago

panda2134 commented 7 months ago

Describe the issue

When trying to use torchvision stdenv complained about not being able to find addAutoPatchelfSearchPath.

That's because on this line we're missing

nativeBuildInputs =  [ pkgs.autoPatchelfHook ];

Additional context

default.nix/shell.nix/flake.nix:

{
  description = "Application packaged using poetry2nix";
  nixConfig.bash-prompt-suffix = "[myApp-dev]> ";

  inputs.flake-utils.url = "github:numtide/flake-utils";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  inputs.poetry2nix = {
    url = "github:nix-community/poetry2nix/master";
    inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs = { self, nixpkgs, flake-utils, poetry2nix }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        # see https://github.com/nix-community/poetry2nix/tree/master#api for more functions and examples.
        inherit (poetry2nix.lib.mkPoetry2Nix { inherit pkgs; }) mkPoetryEnv mkPoetryApplication defaultPoetryOverrides;
        pkgs = nixpkgs.legacyPackages.${system};
      in {
        devShells = {
          myApp = (
            let
              poetryEnv = mkPoetryEnv {
                projectDir = ./.;
                python = pkgs.python311;
                preferWheels = true;
              };
              env = poetryEnv.env.overrideAttrs (_: {
                shellHook = ''
                export LD_LIBRARY_PATH="${pkgs.stdenv.cc.cc.lib}/lib";
                '';
              });
            in env
          );
          default = self.devShells.${system}.myApp;
        };
      });
}

pyproject.toml:

[tool.poetry]
name = "main"
version = "0.1.0"
description = ""
authors = ["Jiangyi Liu"]

[tool.poetry.dependencies]
python = "^3.11"
black = "^24.4.0"
cryptography = "42.0.3"
joblib = "^1.4.0"
matplotlib = "^3.8.4"
numpy = "^1.26.4"
pandas = "^2.2.2"
scikit-learn = "^1.4.2"
scipy = "^1.13.0"
seaborn = "^0.13.2"
torch = {version = "^2.2.2", source = "pytorch-rocm5.7-src"}
pytorch-triton-rocm = {version = "2.2.0", source = "pytorch-rocm5.7-src"}
torchaudio = {version = "^2.2.2", source = "pytorch-rocm5.7-src"}
torchvision = {version = "^0.17.2", source = "pytorch-rocm5.7-src"}

[tool.poetry.dev-dependencies]

[[tool.poetry.source]]
name = "pytorch-rocm5.7-src"
url = "https://download.pytorch.org/whl/rocm5.7"
priority = "explicit"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
DDoSolitary commented 3 months ago

There's already code that adds autoPatchelfHook https://github.com/nix-community/poetry2nix/blob/d11c01e58587e5f21037ed6477465a7f26a32e27/mk-poetry-dep.nix#L131 But it checks for binary wheels by looking for "manylinux" in the wheel file name, which fails to match the package names in pytorch's index image