cryptography 35

[tfmoraes]

This is not a bug report, it's a tip for other with same problem. Since the version 35, cryptography requires rust to build and doesn't respect CRYPTOGRAPHY_DONT_BUILD_RUST. I managed to compile with this overlay:

    overrides = poetry2nix.overrides.withDefaults (self: super: {
      cryptography = super.cryptography.overridePythonAttrs(old:{
        cargoDeps = pkgs.rustPlatform.fetchCargoTarball {
          inherit (old) src;
          name = "${old.pname}-${old.version}";
          sourceRoot = "${old.pname}-${old.version}/src/rust/";
          sha256 = "sha256-tQoQfo+TAoqAea86YFxyj/LNQCiViu5ij/3wj7ZnYLI=";
        };
        cargoRoot = "src/rust";
        nativeBuildInputs = old.nativeBuildInputs ++ (with pkgs.rustPlatform; [
          rust.rustc
          rust.cargo
          cargoSetupHook
        ]);
      });
    });

The only problem is cargoDeps.sha256 must be filled, so I don't know if it's possible to make it dynamic in a way that works for every version of cryptography.

[adisbladis]

I'm hoping that https://github.com/pyca/cryptography/pull/6371 will be able to solve this in a satisfactory manner.

[charmoniumQ]

Given that pyca/cryptography#6371 has been closed, I'm wondering what the direction forward is. It seems like the options are: support only one version of cryptography or hard-code a table of hashes for each release. Any others?

I'd be happy to help if that is necessary.

[pingiun]

For my project I think I'm just gonna try and hardcode it to the nixpkgs master version so it will stay up to date but not with poetry

[adisbladis]

I dropped a comment in the upstream issue https://github.com/pyca/cryptography/pull/6371#issuecomment-998308086.

[charmoniumQ]

In the mean time, how does one workaround this when the hash isn't already hard-coded?

trace: warning: Unknown cryptography version: '40.0.1'. Please update getCargoHash.
error: hash mismatch in fixed-output derivation '/nix/store/znyf7mqmnfyw43sfwpsfcikm93b3d1x5-cryptography-40.0.1-vendor.tar.gz.drv':
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
            got:    sha256-gFfDTc2QWBWHBCycVH1dYlCsWQMVcRZfOBIau+njtDU=

I found poetry2nix/overrides/default.nix

      cryptography =
        let
          getCargoHash = version: {

But how do I compute this hash (what exactly to hash), and how do I override it in my own flake.nix?

[asymmetric]

@charmoniumQ for your first question: the hash is being given to you in the error message above.

To override in your flake.nix, you can add an overlay like:

final: prev: {
  cryptography = prev.cryptography.overridePythonAttrs (old: {
    cargoDeps = pkgs.rustPlatform.fetchCargoTarball {
      src = old.src;
      sourceRoot = "${old.pname}-${old.version}/src/rust";
      name = "${old.pname}-${old.version}";
      # This is what we actually want to patch.
      sha256 = "sha256-6C4N445h4Xf2nCc9rJWpSZaNPilR9GfgbmKvNlSIFqg=";
    };
  });
}

[szicari-farmobile]

Can you provide a simple guideline for someone new to Nix (i.e., me) on where in a flake.nix this override would go?

I tried providing an override to pkgs.${system} at first. After seeing absolutely no change in behavior (i.e., the failure described in this issue continued to happen unchanged), it slowly dawned on me that maybe I'm supposed to override cryptography in the poetry2nix package.

Any help is appreciated! I recognize it is purely due to me being new to Nix, not a lack in the information in the ticket.

Here is how I attempted to add it:

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/23.05";

  outputs = { self, nixpkgs }:
    let
      supportedSystems = [ "x86_64-linux" "x86_64-darwin" "aarch64-linux" "aarch64-darwin" ];
      forAllSystems = nixpkgs.lib.genAttrs supportedSystems;
      overrides = (final: prev: {
        cryptography = prev.cryptography.overridePythonAttrs (old: {
          cargoDeps = pkgs.rustPlatform.fetchCargoTarball {
            src = old.src;
            sourceRoot = "${old.pname}-${old.version}/src/rust";
            name = "${old.pname}-${old.version}";
            # This is what we actually want to patch.
            sha256 = "sha256-38q81vRf8QHR8lFRM2KbH7Ng5nY7nmtWRMoPWS9VO/U=";
          };
        });
      });
      pkgs = forAllSystems (system: nixpkgs.legacyPackages.${system}.extend overrides);
    in
    {
      packages = forAllSystems (system: {
        default = pkgs.${system}.poetry2nix.mkPoetryApplication { projectDir = self; };
      });

      devShells = forAllSystems (system: {
        default = pkgs.${system}.mkShellNoCC {
          packages = with pkgs.${system}; [
            (poetry2nix.mkPoetryEnv { projectDir = self; })
            poetry
          ];
        };
      });
    };
}

[charmoniumQ]

I ended up not needing this override, so I can't tell you exactly.

But overrides is parameter of mkPoetryEnv and mkPoetryApplication, so

default = pkgs.${system}.poetry2nix.mkPoetryApplication {
  projectDir = self;
  inherit overrides;
};
...
(poetry2nix.mkPoetryEnv {
  projectDir = self;
  inherit overrides;
})

[szicari-farmobile]

@charmoniumQ How did you resolve this issue? I would love to not need the override as well. I'm getting the cryptography error every time.

[charmoniumQ]

@szicari-farmobile

There may be two reasons you are hitting this problem and I am not:

One is that I pinned cryptography to 40.0.1 in pyproject.toml (and then ran poetry update and confirmed that the lockfile lists 40.0.1). This is not good long term, because it means you won't get updates on cryptography. Pinning it might downgrade some packages in your environment.

Another possible reason is that your flake is acquiring poetry2nix via nixpkgs which is pinned to the 23.05 release. This means you may not be getting the most recent poetry2nix, which has cryptography up to 41.0.1 at the time of writing (check on that by searching for cryptography here). Check this first before pinning to an unnecessarily old version of cryptography.

To get the most recent poetry2nix, change the first few lines of your flake to:

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/23.05";

  # Get poetry2nix directly from the GitHub source
  inputs.poetry2nixFlake.url = "github:nix-community/poetry2nix";

  # This function accepts an argument called poetry2nixFlake
  outputs = { self, nixpkgs, poetry2nixFlake }:

and replace mkApplication from poetry2nixFlake.legacyPackages.${system}.mkApplication (see end for a cleaner way).

The first time you build the flake, Nix will write the current revision of poetry2nix into flake.lock and you will be pinned on that version. You can update that pin by running nix flake update poetry2nixFlake, but always take caution to test for software regressions in case this update breaks your code. This will update the ref in the flake.lock to point to the newest revision of poetry2nix.

---

Some advice you didn't ask for:

The point of Nix is to have reproducible environments (hence the flake.lock). You can specifically request an update (nix flake update $flakeInputName), but by definition the updated environment will not be a bit-for-bit reproduction of the prior environment, so do that carefully.

Another concept is distributed package management. Every flake (poetry2nix flake, your flake, etc.) can define its own set of packages, which are no different than those in nixpkgs (which is exposed as normal flake), kind of like a PPA in Debian/Ubuntu.

If I can offer one more suggestion, I think it's simpler use numtide's flake-utils to map over all systems.

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/23.05";
    flake-utils.url = "github:numtide/flake-utils";
    poetry2nixFlake.url = "github:nix-community/poetry2nix";
  };
  outputs = { self, nixpkgs, poetry2nixFlake, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      # This fn gets called once for every system.
      let
        pkgs = nixpkgs.legacyPackages.${system};
        poetry2nix = poetry2nixFlake.legacyPackages.${system};
        # all system-specific up stuff here.
      in
      rec {
        # eachDefaultSystem expands packages.default -> packages.${system}.default
        packages.default = poetry2nix.mkApplications { ... };
        devShells.default = pkgs.mkShellNoCC { ... };
      }
    );
}

which you can also get to by running nix flake init -t github:numtide/flake-utils.

[l0b0]

I was hoping to simply override getCargoHash, but I can't seem to get that working. The build seems to simply ignore the following:

cryptography = super.cryptography.overridePythonAttrs (old: {
  getCargoHash = "sha256-38q81vRf8QHR8lFRM2KbH7Ng5nY7nmtWRMoPWS9VO/U=";
});

Or even

cryptography = super.cryptography.overridePythonAttrs (old: {
  sha256 = "sha256-38q81vRf8QHR8lFRM2KbH7Ng5nY7nmtWRMoPWS9VO/U=";
});

Presumably it's not that simple to override something in the let section of an override, if it's possible at all.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/poetry2nix-mismatch-cryptography-sha256-but-unable-to-modify/33897/1