search

pola-rs / r-polars

Bring polars to R
https://pola-rs.github.io/r-polars/
Other
415 stars 36 forks source link

building with nix flake gives missing dependency error #226

Closed peddie closed 1 year ago

peddie commented 1 year ago

I tried to run

nix develop github:pola-rs/r-polars

as suggested in https://github.com/pola-rs/r-polars/pull/54 (the PR adding the flake). I got an error:

error: A hash was specified for arrow2-0.16.0, but there is no corresponding git dependency.

I get the same error if I add this repo as an input to my own flake which depends on r-polars.

Indeed, apart from an outputHash, I see no reference to the arrow2 package in the flake.nix file. Is there a better way to install r-polars via nix?

eitsupi commented 1 year ago

@Sicheng-Pan Any thoughts?

Sicheng-Pan commented 1 year ago

Well it seems that some dependencies have been updated since my last PR, but the flake file is not (there are some hashes that have to be manually updated since some dependencies are not fixed in rust toml). I have updated it on my fork and I'll make a PR for this.

peddie commented 1 year ago

Thanks @Sicheng-Pan, the command works when I point it at your fork. I look forward to seeing it merged.

peddie commented 1 year ago

@Sicheng-Pan , I tried to bring in r-polars.packages.${system}.default as a dependency in my flake, and I received

@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/3l0jjnhwcd0cmv39vwb666xqf0xc9xpm-source
source root is source
Executing cargoSetupPostUnpackHook
unpacking source archive /nix/store/x7f7vv4fbgffsacj3kq1zzyg7li0r1al-cargo-vendor-dir
Finished cargoSetupPostUnpackHook
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
Executing cargoSetupPostPatchHook
Validating consistency between /build/source/src/rust/Cargo.lock and /build/cargo-vendor-dir/Cargo.lock
Finished cargoSetupPostPatchHook
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
@nix { "action": "setPhase", "phase": "buildPhase" }
building
@nix { "action": "setPhase", "phase": "installPhase" }
installing
* installing *source* package 'polars' ...
** using staged installation
** libs
rm -Rf polars.so ./rust/target/release/libr_polars.a entrypoint.o
/nix/store/bfbp3ypd9nm3fapz634gvvs738blrl0y-gcc-wrapper-12.2.0/bin/cc -I"/nix/store/j48jssi5nfz5v4hkajkiwwmksyiingz4-R-4.2.2/lib/R/include" -DNDEBUG      -fpic  -g -O2  -c entrypoint.c -o entrypoint.o
cargo build --lib --release --manifest-path="./rust/Cargo.toml"
error: failed to get `extendr-api` as a dependency of package `r-polars v0.1.0 (/build/source/src/rust)`

Caused by:
  failed to load source for dependency `extendr-api`

Caused by:
  Unable to update https://github.com/rpolars/extendr?branch=errorhandling#d5eba3af

Caused by:
  failed to update replaced source https://github.com/rpolars/extendr?branch=errorhandling#d5eba3af

Caused by:
  failed to parse manifest at `/build/cargo-vendor-dir/polars-io-0.28.0/Cargo.toml`

Caused by:
  error inheriting `version` from workspace root manifest's `workspace.package.version`

Caused by:
  failed to find a workspace root
make: *** [Makevars:13: rust/target/release/libr_polars.a] Error 101
ERROR: compilation failed for package 'polars'
* removing '/nix/store/gw3cybgy5qz3q1c45bb4126q3wnay69l-r-polars/library/polars'

Is this related to the packaging changes? Am I doing something wrong?

Sicheng-Pan commented 1 year ago

This (error inheriting Rust workspace version) seems to be an old bug triggered when building Rust projects on Nix, and this should have been fixed in recent commits on nixpkgs-unstable. Are you using the latest nixpkgs for your flake?

peddie commented 1 year ago

If I update to the latest nixpkgs-unstable (8d4d822bc0 2023-06-01) and try again, I get the same error, but pointing to polars-arrow-0.28.0 instead of polars-io-0.28.0 as in the above log. I'm not sure whether this difference is significant.

Sicheng-Pan commented 1 year ago

This is strange. nix shell github:pola-rs/r-polars works fine on my side. Does this work on your side? Also if possible would you please share (parts of) your flake so that I can replicate the error on my side?

peddie commented 1 year ago

Yes -- as I mentioned above, nix develop github:pola-rs/r-polars works fine (as does nix shell). The problem only occurs when I try to depend on the library output. I will post a minimal example as soon as I'm able.

peddie commented 1 year ago

flake.nix:

{
  description = "R development shell.";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils/main";
    r-polars.url = "github:pola-rs/r-polars";
  };

  outputs = { self, nixpkgs, flake-utils, r-polars, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        packageName = "R-dev-shell";
      in rec {
        devShell = pkgs.mkShell {
          buildInputs = with pkgs; [
            r-polars.packages.${system}.default
          ];
        };
        packages.default = devShell;

      });

}

flake.lock:

{
  "nodes": {
    "fenix": {
      "inputs": {
        "nixpkgs": "nixpkgs_2",
        "rust-analyzer-src": "rust-analyzer-src"
      },
      "locked": {
        "lastModified": 1682038649,
        "narHash": "sha256-HwGwWLMKdIT24xhDf+mRoCehA8yUlLmuJgS9JeMt4IM=",
        "owner": "nix-community",
        "repo": "fenix",
        "rev": "37b3a6dad6d6060bd305eb7d3628d3b476c87bb6",
        "type": "github"
      },
      "original": {
        "owner": "nix-community",
        "ref": "monthly",
        "repo": "fenix",
        "type": "github"
      }
    },
    "flake-utils": {
      "inputs": {
        "systems": "systems"
      },
      "locked": {
        "lastModified": 1681202837,
        "narHash": "sha256-H+Rh19JDwRtpVPAWp64F+rlEtxUWBAQW28eAi3SRSzg=",
        "owner": "numtide",
        "repo": "flake-utils",
        "rev": "cfacdce06f30d2b68473a46042957675eebb3401",
        "type": "github"
      },
      "original": {
        "owner": "numtide",
        "ref": "main",
        "repo": "flake-utils",
        "type": "github"
      }
    },
    "flake-utils_2": {
      "locked": {
        "lastModified": 1676283394,
        "narHash": "sha256-XX2f9c3iySLCw54rJ/CZs+ZK6IQy7GXNY4nSOyu2QG4=",
        "owner": "numtide",
        "repo": "flake-utils",
        "rev": "3db36a8b464d0c4532ba1c7dda728f4576d6d073",
        "type": "github"
      },
      "original": {
        "owner": "numtide",
        "repo": "flake-utils",
        "type": "github"
      }
    },
    "nixpkgs": {
      "locked": {
        "lastModified": 1685591878,
        "narHash": "sha256-Ib3apaLqIFkZb94q6Q214DXrz0FnJq5C7usywTv63og=",
        "owner": "nixos",
        "repo": "nixpkgs",
        "rev": "8d4d822bc0efa9de6eddc79cb0d82897a9baa750",
        "type": "github"
      },
      "original": {
        "owner": "nixos",
        "ref": "nixpkgs-unstable",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "nixpkgs_2": {
      "locked": {
        "lastModified": 1677932085,
        "narHash": "sha256-+AB4dYllWig8iO6vAiGGYl0NEgmMgGHpy9gzWJ3322g=",
        "owner": "nixos",
        "repo": "nixpkgs",
        "rev": "3c5319ad3aa51551182ac82ea17ab1c6b0f0df89",
        "type": "github"
      },
      "original": {
        "owner": "nixos",
        "ref": "nixos-unstable",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "nixpkgs_3": {
      "locked": {
        "lastModified": 1677932085,
        "narHash": "sha256-+AB4dYllWig8iO6vAiGGYl0NEgmMgGHpy9gzWJ3322g=",
        "owner": "nixos",
        "repo": "nixpkgs",
        "rev": "3c5319ad3aa51551182ac82ea17ab1c6b0f0df89",
        "type": "github"
      },
      "original": {
        "owner": "nixos",
        "ref": "nixos-unstable",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "r-polars": {
      "inputs": {
        "fenix": "fenix",
        "flake-utils": "flake-utils_2",
        "nixpkgs": "nixpkgs_3"
      },
      "locked": {
        "lastModified": 1685613136,
        "narHash": "sha256-CfYcKD2Be06Rm9UweJbcI5tIIfd5cD6MVNWVelZ5gIY=",
        "owner": "pola-rs",
        "repo": "r-polars",
        "rev": "3903d33f358b1b896eea8eab98c70b0ca3af4860",
        "type": "github"
      },
      "original": {
        "owner": "pola-rs",
        "repo": "r-polars",
        "type": "github"
      }
    },
    "root": {
      "inputs": {
        "flake-utils": "flake-utils",
        "nixpkgs": "nixpkgs",
        "r-polars": "r-polars"
      }
    },
    "rust-analyzer-src": {
      "flake": false,
      "locked": {
        "lastModified": 1678033668,
        "narHash": "sha256-Njlus+vY3N++qWE0JXrGjwcXY2QDFuOV/7NruBBMETY=",
        "owner": "rust-lang",
        "repo": "rust-analyzer",
        "rev": "0a956ec9326eca09725d64d9f1b63896f93505d1",
        "type": "github"
      },
      "original": {
        "owner": "rust-lang",
        "ref": "nightly",
        "repo": "rust-analyzer",
        "type": "github"
      }
    },
    "systems": {
      "locked": {
        "lastModified": 1681028828,
        "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
        "owner": "nix-systems",
        "repo": "default",
        "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
        "type": "github"
      },
      "original": {
        "owner": "nix-systems",
        "repo": "default",
        "type": "github"
      }
    }
  },
  "root": "root",
  "version": 7
}

@Sicheng-Pan these files show the same problem.

Sicheng-Pan commented 1 year ago

@peddie Your flake and lock will indeed produce the error. However, it seems that the dependencies of your r-polars input is older than expected. Please use nix flake update to update all inputs for your flake (or at least the dependencies of r-polars). It works after the update on my end.

peddie commented 1 year ago

Thanks for your help; I misunderstood the nature of the original problem. I'm able to pull this in as a dependency now.