Package set generator for Nixpkgs

[FRidh]

I would like to have a tool that generates a package set, but does not use pip to resolve dependencies. Instead, it should always use either the latest version or the version that was explicitly requested:

1. A Nix expression listing requested packages along with a maintainer for each and optionally a version.
2. Tool that takes the Nix expression, uses the PyPI API to find a file and download it, extract relevant info.
3. Recursively extract this info for transitive dependencies.
4. Write output as a single big file.

Because generating the whole package set takes time, and because we will want to tune versions, I think fetching and extracting (step 2) should be done in a nix-build. This will be even slower, especially when ran only once, but I think it may take time tuning versions making it worth it. For each package the nix-build will generate a JSON file. The tool will obviously need to keep track of all files generated.

Furthermore, it should be possible to perform only minor or patch updates. That effectively means reading the earlier generated expression, creating constraints based on that, and injecting it into step 1.

What do you think?

[costrouc]

So if I understand correctly.

{
  packages = [
     { name = "requests" ; maintainers = [ "costrouc" ]; }
     { name = "six"; maintainers = [ "FRidh" ]; version = "1.11.0"; }
  ];
}

This would generate a nix expression to build all of these packages with the names normalized meta information auto generated. Then patch can be easily applied over this for each specific package?

How does this handle packages that depend on shared libraries (would this be in the patch)?

[costrouc]

@FRidh I have put some work into this issue. So now python-package-init has the ability to add new packages to nixpkgs (pkgs/top-level/python-modules.nix and pkgs/development/python-modules/<package-name>/default.nix).

For example

python-package-init cattrs --nixpkgs-root=<path to nixpkgs root>

This generates the following diff

From c918f8385ebc8222195d7e9542d61d3a2dc958af Mon Sep 17 00:00:00 2001
From: Chris Ostrouchov <chris.ostrouchov@gmail.com>
Date: Fri, 26 Jul 2019 16:32:31 -0400
Subject: [PATCH] simple commit

---
 .../python-modules/cattrs/default.nix         | 52 +++++++++++++++++++
 pkgs/top-level/python-packages.nix            |  2 +
 2 files changed, 54 insertions(+)
 create mode 100644 pkgs/development/python-modules/cattrs/default.nix

diff --git a/pkgs/development/python-modules/cattrs/default.nix b/pkgs/development/python-modules/cattrs/default.nix
new file mode 100644
index 00000000000..94c73eae867
--- /dev/null
+++ b/pkgs/development/python-modules/cattrs/default.nix
@@ -0,0 +1,52 @@
+{ lib
+, buildPythonPackage
+, fetchPypi
+, attrs
+, functools32
+, singledispatch
+, enum34
+, typing
+}:
+
+buildPythonPackage rec {
+  pname = "cattrs";
+  version = "0.9.0";
+
+  src = fetchPypi {
+    inherit pname version;
+    sha256 = "2232a568e079f30601de63839f4101283cb594c8719f5311bede25f89d0e09b7";
+  };
+
+  # # Package conditions to handle
+  # # might have to sed setup.py and egg.info in patchPhase
+  # # sed -i "s/<package>.../<package>/"
+  # attrs >= 17.3
+  # functools32 >= 3.2.3; python_version<'3.0'
+  # singledispatch >= 3.4.0.3; python_version<'3.0'
+  # enum34 >= 1.1.6; python_version<'3.0'
+  # typing >= 3.5.3; python_version<'3.0'
+  # # Extra packages (may not be necessary)
+  # bumpversion # dev
+  # wheel # dev
+  # watchdog # dev
+  # flake8 # dev
+  # tox # dev
+  # coverage # dev
+  # Sphinx # dev
+  # pytest # dev
+  # hypothesis # dev
+  propagatedBuildInputs = [
+    attrs
+    functools32
+    singledispatch
+    enum34
+    typing
+  ];
+
+  meta = with lib; {
+    description = "Composable complex class support for attrs";
+    homepage = https://github.com/Tinche/cattrs;
+    license = licenses.mit;
+    # maintainers = [ maintainers. ];
+  };
+}
\ No newline at end of file
diff --git a/pkgs/top-level/python-packages.nix b/pkgs/top-level/python-packages.nix
index 4ef5f820c0c..cb9b8e43d8c 100644
--- a/pkgs/top-level/python-packages.nix
+++ b/pkgs/top-level/python-packages.nix
@@ -434,6 +434,8 @@ in {

   braintree = callPackage ../development/python-modules/braintree { };

+  cattrs = callPackage ../development/python-modules/cattrs { };
+
   django-sesame = callPackage ../development/python-modules/django-sesame { };

   breathe = callPackage ../development/python-modules/breathe { };
-- 
2.22.0

[makefu]

That means that the issue is closed now or should we keep it open? cc @costrouc @FRidh

[makefu]

lets close this