Packaging Request: Hydrus Network

[rendeko]

Been trying to package this one all month with no luck, as it uses both Qt wrapping and python modules that are broken in the Nix repo.

A personal booru-style media tagger that can import files and tags from your hard drive and popular websites.

Metadata

• homepage URL: http://hydrusnetwork.github.io/hydrus/
• source URL: https://github.com/hydrusnetwork/hydrus/
• license: WFPL
• platforms: Linux/Windows/OSX

[deliciouslytyped]

Just your luck that I happened to see this, I have some work on this lying around somewhere but it might have been rather bad. Also I don't know if it was a filesystem issues or what, but I stopped trying to use it because I had some major performance problems with the initial processing of the tag database or whatever, and it would have taken forever to finish. I'll post what I have when I find it. Also, don't look at the hydrus git history. :P

[deliciouslytyped]

This doesn't look like it works, but it's what I have:

#waiting on https://github.com/NixOS/nixpkgs/issues/54235
{nixpkgs ? import <nixpkgs>}:

with nixpkgs;

let
  hydrus-git = pkgs.fetchFromGithub {
    owner = "hydrusnetwork";
    repo = "hydrus";
    rev = "99478235dc05e151c464c47fbdcbea78a0ecc44a";
    sha256 = "0";
    };
in
  pkgs.python37.withPackages (pkgs: with pkgs; [
    beautifulsoup4 html5lib lxml nose numpy opencv-python six Pillow psutil PyOpenSSL PyYAML requests Send2Trash service_identity twisted
    #optionals#TODO
    lz4 pylzma pysocks matplotlib 
    #builds #mock httmock pyinstaller
    ])

It's probably worth looking into packaging it with poetry2nix now. I know i definitely had hydrus working in some form. It may have just been pip. I'm not sure.

That "waiting on" is done now because wxPython 4 is packaged.

[rendeko]

Glad someone else is interested! I managed to get a decent nix-shell going by just installing every package with pip3 in postShellHook, but Qt was struggling to find libraries and the method to wrap Qt apps for packages just makes no sense to me.

Any method of it working would be amazing as its current use is in my Windows VM.

[deliciouslytyped]

Post what you have?

[rendeko]

For reference sake: I am 100% not a packaging person.

with import <nixpkgs> {};

mkShell {
    name = "hydrus";

    buildInputs = with python37Packages; [
        python3
        venvShellHook
        glib
        zlib
        libkrb5
        libGL
    ];

    # environmental variables
    QT_DEBUG_PLUGINS = 1;

    venvDir = "venv37";

    nativeBuildInputs = [ qt5.wrapQtAppsHook ];

    postShellHook = ''  
        export LD_LIBRARY_PATH=${lib.makeLibraryPath [ glib stdenv.cc.cc.lib libkrb5 gcc libGL ]}

        touch somefile
        chmod +x somefile
        makeQtWrapper somefile sourceme
        sed -i -e '/exec/d' sourceme
        source sourceme

        unset SOURCE_DATE_EPOCH # allow pip to install wheels
        pip3 install --upgrade pip setuptools wheel
        pip3 install beautifulsoup4
        pip3 install chardet
        pip3 install cloudscraper
        pip3 install html5lib
        pip3 install lxml
        pip3 install lz4
        pip3 install nose
        pip3 install numpy
        pip3 install opencv-python-headless
        pip3 install Pillow
        pip3 install psutil
        pip3 install pylzma
        pip3 install pyOpenSSL
        pip3 install PySide2
        pip3 install shiboken2
        pip3 install PyQt5
        pip3 install PySocks
        pip3 install python-mpv
        pip3 install PyYAML
        pip3 install QtPy
        pip3 install requests
        pip3 install Send2Trash
        pip3 install service-identity
        pip3 install six
        pip3 install Twisted
    '';

}

[jorsn]

In case the broken python packages are connected with qt5, see #98185. I packaged a quite complex application (python+qt5+libreoffice+vlc+gstreamer) here in #91291. Feel free to take it as inspiration. Albeit, e.g. qutebrowser might be a somewhat simpler example for python+qt5.

Basically, you should probably use libsForQt514.callPackage to call import the package file, and therein use mkDerivationWith buildPythonPackage (for libraries) or mkDerivationWith buildPythonApplication. If you read the respecting sections in the nixpkgs manual and especially about wrapping of qt, glib and python programs then packaging should not be too difficult anymore. You'll probably have to set something like this:

{ mkDerivationWith, python3Packages, wrapGAppsHook, … }:

mkDerivationWith python3Packages.buildPythonPackage {

…

dontWrapQtApps = true;
dontWrapGApps = true;

nativeBuildInputs = [ wrapGAppsHook ];

makeWrapperArgs = [
  "\${gappsWrapperArgs[@]}"
  "\${qtWrapperArgs[@]}"
];

postFixup = ''
    wrapPythonPrograms
'';
}

as I do in #91291.