luaPackages.lgi fails to find cairo

[pinpox]

Describe the bug

As briefly discussed here luaPackages.lgi recently started to produced errors, as it is no longer able to find cairo. I'm trying to run this application with:

nix run "github:pinpox/wallpaper-generator -- harmonograph -o harmonograph.png

The application used to run fine, now I see this error:

(process:2196021): GLib-GObject-WARNING **: 14:46:54.687: cannot retrieve class for invalid (unclassed) type '<invalid>'

(process:2196021): GLib-GObject-WARNING **: 14:46:54.689: cannot retrieve class for invalid (unclassed) type '<invalid>'
/nix/store/j3v13kmaby3w18sb4g8xdz2qpmarvv66-lua-5.2.4/bin/lua: ...7prps7g8hidljnjl-lua-5.2.4-env/share/lua/5.2/lgi/ffi.lua:84: bad argument #1 to 'cast' (lgi.record expected, got nil)
stack traceback:
        [C]: in function 'cast'
        ...7prps7g8hidljnjl-lua-5.2.4-env/share/lua/5.2/lgi/ffi.lua:84: in function 'load_enum'
        ...ljnjl-lua-5.2.4-env/share/lua/5.2/lgi/override/cairo.lua:45: in main chunk
        [C]: in function 'require'
        ...g8hidljnjl-lua-5.2.4-env/share/lua/5.2/lgi/namespace.lua:183: in function <...g8hidljnjl-lua-5.2.4-env/share/lua/5.2/lgi/namespace.lua:155>
        (...tail calls...)
        ...wallpaper-generator-1.0/bin/.wallpaper-generator-wrapped:15: in main chunk
        [C]: in ?

I suspect this is issue happens since this commit

Steps To Reproduce

Steps to reproduce the behavior:

1. nix run ".#" -- harmonograph -o harmonograph.png
2. ...
3. ...

Expected behavior

Application should run and produce an image

Additional context

Issue with more logs has been discussed here

Notify maintainers

@teto

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.62, NixOS, 21.11 (Porcupine)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.4pre20210908_3c56f62`
 - channels(root): `""`
 - channels(pinpox): `"home-manager"`
 - nixpkgs: `/nix/store/yfjhaanny02sk592mnw2ddr87qalki49-source`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
-  pkgs.luaPackages.lgi
# a list of nixos modules affected by the problem
module:

[teto]

I've bumped the flake and tried nix run:

       at /nix/store/1y8fdj2a951by77n3vfpi6jyhmj2fvyg-source/pkgs/top-level/lua-packages.nix:56:22:

           55|   getLuaPath = drv: luaLib.getPath drv (luaLib.luaPathList lua.luaversion) ;
           56|   getLuaCPath = drv: luaLib.getPath drv (luaLib.luaCPathList lua.luaversion) ;
             |                      ^

Indeed there is an error in nixpkgs but no package is using this function so the issue was not detected.

Regarding the flake, you should move cairo/gobject-introspection to buildInputs and wrapLuaProgram should automatically wrap the executable in $out/bin with the correct LUA_PATH / LUA_CPATH. but it is not used in nixpkgs so I am not sure how well that works :/ I have some local changes to lua I want to upstream, I can probably have a look at this later this week if you still can't make it work.

[pinpox]

I added cairo and gobject-introspection to buildInputs and tried to use wrapLuaPrograms (I guess you meant the plural, with 's' ? ) like so:

            buildInputs = [
              pkgs.luaPackages.wrapLua
              pkgs.cairo
              pkgs.gobject-introspection
              lua-with-pkgs
            ];

            installPhase = ''
              install -Dm755 main.lua $out/bin/wallpaper-generator
              cp -r generators $out/bin/generators
              wrapLuaPrograms $out/bin/wallpaper-generator
            '';

This results in an error, it can't find addToLuaPath

✦ » nix run ".#" -- harmonograph -o harmonograph.png
warning: Git tree '/home/pinpox/code/github.com/pinpox/wallpaper-generator' is dirty
error: builder for '/nix/store/6xq1r96nyxb8q1a90smps7yhi5dm7v0c-wallpaper-generator-1.0.drv' failed with exit code 127;
       last 9 log lines:
       > unpacking sources
       > unpacking source archive /nix/store/qs7g21xxgmh7ypcashihdnpk3i2cgsfg-source
       > source root is source
       > patching sources
       > configuring
       > no configure script, doing nothing
       > glibPreInstallPhase
       > installing
       > /nix/store/zb01f258cvr5zfwrkkqfqs27iyy35p9v-hook/nix-support/setup-hook: line 24: addToLuaPath: command not found
       For full logs, run 'nix log /nix/store/6xq1r96nyxb8q1a90smps7yhi5dm7v0c-wallpaper-generator-1.0.drv'.

Not sure if I'm using it wrong, couldn't any good examples for it. If you have he time, it would be great if you can help me debug this, I'm at a loss here.

Even more puzzeling, the same flake works perfectly inside an isolated docker container with nix:

docker run --rm -it nixpkgs/nix-flakes bash -c "nix run "github:pinpox/wallpaper-generator" -- harmonograph"

Any ideas what the difference is?

[teto]

I will help since there are issues with nixpkgs but I am running out of FOSS credits xD I will try to complete https://github.com/NixOS/nixpkgs/pull/139616 first that should fix the getLuaPath, and will look afterwards into the wrapLua* problem, but I think I know why it fails.

[pinpox]

I saw that #139616 was merged, but getLuaPath seems to be still broken. Any update on this? Let me know if I can help out

[teto]

getLuaPath was fixed by another PR, did that help ?

also I noticed there is a patch applied to lgi, maybe it went out of sync ?

  lgi = prev.lib.overrideLuarocks prev.lgi (drv: {
    nativeBuildInputs = [
      pkgs.pkg-config
    ];
    buildInputs = [
      pkgs.glib
      pkgs.gobject-introspection
    ];
    patches = [
      (pkgs.fetchpatch {
        name = "lgi-find-cairo-through-typelib.patch";
        url = "https://github.com/psychon/lgi/commit/46a163d9925e7877faf8a4f73996a20d7cf9202a.patch";
        sha256 = "0gfvvbri9kyzhvq3bvdbj2l6mwvlz040dk4mrd5m9gz79f7w109c";
      })
    ];

    # there is only a rockspec.in in the repo, the actual rockspec must be generated
    preConfigure = ''
      make rock
    '';
  });

[pinpox]

Sadly no. The error still persists:

×  nix run
warning: Git tree '/home/pinpox/code/github.com/pinpox/wallpaper-generator' is dirty

(process:2798810): GLib-GObject-WARNING **: 10:04:36.263: cannot retrieve class for invalid (unclassed) type '<invalid>'

(process:2798810): GLib-GObject-WARNING **: 10:04:36.265: cannot retrieve class for invalid (unclassed) type '<invalid>'
/nix/store/pl60bqsgp9klgf2aqzgv24pdzwdynysc-lua-5.2.4/bin/lua: ...1ac2yj52607kpqs9-lua-5.2.4-env/share/lua/5.2/lgi/ffi.lua:84: bad argument #1 to 'cast' (lgi.record expected, got nil)
stack traceback:
        [C]: in function 'cast'
        ...1ac2yj52607kpqs9-lua-5.2.4-env/share/lua/5.2/lgi/ffi.lua:84: in function 'load_enum'
        ...kpqs9-lua-5.2.4-env/share/lua/5.2/lgi/override/cairo.lua:45: in main chunk
        [C]: in function 'require'
        ...52607kpqs9-lua-5.2.4-env/share/lua/5.2/lgi/namespace.lua:183: in function <...52607kpqs9-lua-5.2.4-env/share/lua/5.2/lgi/namespace.lua:155>
        (...tail calls...)
        ...wallpaper-generator-1.0/bin/.wallpaper-generator-wrapped:4: in main chunk
        [C]: in ?

[stale[bot]]

I marked this as stale due to inactivity. → More info

[teto]

Does it still fail ? there were several imporvements recently ?

[pinpox]

I'm not sure it is the same problem, havn't touched the code in a while. But after updating the flake.lock file, I get this error now from the software that I was trying to run:

❯ nix run 'github:pinpox/wallpaper-generator' 
/nix/store/f4kccbmzw55jcx17kjjaa05v8kmxz5v6-lua-5.2.4/bin/lua: ...4iqw50kc3xsmh8dz-lua-5.2.4-env/share/lua/5.2/lgi/ffi.lua:31: attempt to index local 'gobject' (a boolean value)
stack traceback:
    ...4iqw50kc3xsmh8dz-lua-5.2.4-env/share/lua/5.2/lgi/ffi.lua:31: in main chunk
    [C]: in function 'require'
    ...qw50kc3xsmh8dz-lua-5.2.4-env/share/lua/5.2/lgi/class.lua:21: in main chunk
    [C]: in function 'require'
    ...kc3xsmh8dz-lua-5.2.4-env/share/lua/5.2/lgi/namespace.lua:18: in main chunk
    [C]: in function 'require'
    ...iqw50kc3xsmh8dz-lua-5.2.4-env/share/lua/5.2/lgi/init.lua:47: in main chunk
    [C]: in function 'require'
    ...vk324iqw50kc3xsmh8dz-lua-5.2.4-env/share/lua/5.2/lgi.lua:19: in main chunk
    [C]: in function 'require'
    ...wallpaper-generator-1.0/bin/.wallpaper-generator-wrapped:2: in main chunk
    [C]: in ?

I'm not sure it is the same problem though, maybe you know what is missing or if the error is related.

[blurgyy]

Also hitting the "attempt to index local 'gobject' (a boolean value)" error, this error made it into branch release-24.05. Minimal reproduction example:

$ nix build nixpkgs/release-24.05#lua51Packages.lgi
$ nix run nixpkgs/release-24.05#lua51Packages.lua
Lua 5.1.5  Copyright (C) 1994-2012 Lua.org, PUC-Rio
> package.cpath = "./result/lib/lua/5.1/?/corelgilua51.so;" .. package.path
> package.path = "./result/share/lua/5.1/?.lua;./result/share/lua/5.1/?/init.lua;" .. package.path
> require('lgi')
./result/share/lua/5.1/lgi/ffi.lua:31: attempt to index local 'gobject' (a boolean value)
stack traceback:
        ./result/share/lua/5.1/lgi/ffi.lua:31: in main chunk
        [C]: in function 'require'
        ./result/share/lua/5.1/lgi/class.lua:21: in main chunk
        [C]: in function 'require'
        ./result/share/lua/5.1/lgi/namespace.lua:18: in main chunk
        [C]: in function 'require'
        ./result/share/lua/5.1/lgi/init.lua:47: in main chunk
        [C]: in function 'require'
        ./result/share/lua/5.1/lgi.lua:19: in main chunk
        [C]: in function 'require'
        stdin:1: in main chunk
        [C]: ?
>

It was fine in the last release, a machine of mine using a 4-month-old nixpkgs version also works fine, didn't try to bisect, but here is the log when it runs without error on branch release-23.11:

$ nix build nixpkgs/release-23.11#lua51Packages.lgi
$ nix run nixpkgs/release-23.11#lua51Packages.lua
> package.cpath = "./result/lib/lua/5.1/?/corelgilua51.so;" .. package.path
> package.path = "./result/share/lua/5.1/?.lua;./result/share/lua/5.1/?/init.lua;" .. package.path
> require('lgi')
>

[stefano-m]

@blurgyy and @pinpox , it appears that the problem is that the GObject typelib file has been moved from gobject-introspection to glib in NixOS 24.05.

gobject-introspection 1.80 does not have the Gio, GLib, GModule and GObject typelib files: they are all in the glib package.

I am not 100% sure of why lgi does not find it because both packages are in lgi's buildInputs

https://github.com/NixOS/nixpkgs/blob/3f6e6d19c2ce6c4c62eb75c452bf01e8e3b96efa/pkgs/development/lua-modules/overrides.nix#L221-L224

Maybe they should be propagatedBuildInputs, but that would make the derivation probably too big.

I tried a very dirty workaround by adding GLib's girepository to the GI_TYPELIB_PATH env var like below

GI_TYPELIB_PATH=${lib.getLib flakePkgs.gobject-introspection}/lib/girepository-1.0:${lib.getLib flakePkgs.glib}/lib/girepository-1.0/
export GI_TYPELIB_PATH

And with this the below Lua script works

ffi = require 'lgi.ffi'
print("FFI")
print(ffi.types.boolean)  -- userdata: 0x4750f8
print("DONE")

It may be worth opening a brand new issue about this though.