The module is not installable on CentOS

melezhik commented 4 years ago

Hi! Even though libgd is installed, the installation process throws an error:

18:12:14 06/28/2020 [bash: zef install .] ===> Testing: GD:ver<0.0.1>:auth<Henrique Dias>
18:12:19 06/28/2020 [bash: zef install .] [GD] Cannot locate native library 'libgd.so': libgd.so: cannot open shared object file: No such file or directory

http://rakudist.raku.org/sparky/report/centos/430

gd information

[root@be9a4df17b1e /]# yum list installed | grep gd
gd.x86_64                           2.2.5-6.el8                           @AppStream 

[root@be9a4df17b1e /]# rpm -ql gd
/usr/lib/.build-id
/usr/lib/.build-id/aa
/usr/lib/.build-id/aa/71d959aab08389a726611ed34d93f9580f6ca9
/usr/lib64/libgd.so.3
/usr/lib64/libgd.so.3.0.5
/usr/share/licenses/gd
/usr/share/licenses/gd/COPYING

melezhik commented 4 years ago

I guess it is because inside source code there is no reference to GD version, it's just gd library ...

sub gdImageGif(GD::Image, GD::File)
        is native('gd') { ... };

cc @niner

melezhik commented 4 years ago

I also had to patch META6.json file - https://github.com/melezhik/raku-GD/blob/master/META6.json#L9-L14 on my fork, to make GD module native dependencies installable ...

jonathanstowe commented 4 years ago

Yep, this is one of a number of things I am working on.

Thanks,

melezhik commented 4 years ago

@jonathanstowe please be aware that version 3 works for recent centos, for old ones it needs to be version2:

dnf --quiet repoquery --whatprovides 'libgd.so.2()(64bit)'
gd-0:2.0.35-26.el7.x86_64

Though I don't think that META6 syntax allows to have versions conditional on CentOS version ...

jonathanstowe commented 4 years ago

Though, actually the meta thing is relatively unimportant, it's the native declaration that is more important.

niner commented 4 years ago

It does allow for that. Please have a look at Inline::Python's META6.json: https://github.com/niner/Inline-Python/blob/master/META6.json

The dependency doesn't sound distro dependent though in this case. Sounds like the module just supports multiple versions of the library.

jonathanstowe commented 4 years ago

Yeah, it just needs to support more than one version of libgd without requiring a devel package, I have a hack for that somewhere.

melezhik commented 4 years ago

@niner , Inline::Pythin has an example of split by different distros:

{
                    "from" : "bin",
                    "name" : {
                        "by-distro.name" : {
                            "macosx" : "python2.7-config",
                            "debian" : "python2.7-config",
                            "" : "python2-config"
                        }
                    }
                }

How does one split by different versions of CentOS?

melezhik commented 4 years ago

@niner:

The dependency doesn't sound distro dependent though in this case

does it?

Compare:

CentOS8 only has gd so version 3

[root@be9a4df17b1e /]# lsb_release -d
Description:    CentOS Linux release 8.1.1911 (Core) 
[root@be9a4df17b1e /]#  dnf --quiet repoquery --whatprovides 'libgd.so.2()(64bit)' --arch x86_64 } wc -l
0
[root@be9a4df17b1e /]#  dnf --quiet repoquery --whatprovides 'libgd.so.3()(64bit)' --arch x86_64
gd-0:2.2.5-6.el8.x86_64

CentOS7 has only so version2

[melezhik@localhost raku-native-deps]$ lsb_release -d
Description:    CentOS Linux release 7.7.1908 (Core)
[melezhik@localhost raku-native-deps]$ dnf --quiet repoquery --whatprovides 'libgd.so.2()(64bit)'
gd-0:2.0.35-26.el7.x86_64
[melezhik@localhost raku-native-deps]$ dnf --quiet repoquery --whatprovides 'libgd.so.3()(64bit)' | wc -l
0

jonathanstowe commented 4 years ago

right, but the particular version that is shipped with a particular OS version is not actually important, what is important is the versions that it will work with and needs to be be specified to the native trait for it to work at all without installing a devel package, That's the actual issue, The meta thing is just a nice to have

melezhik commented 4 years ago

and needs to be be specified to the native trait

@jonathanstowe what is that?

The meta thing is just a nice to have

after some discussion, especially that - https://github.com/ugexe/zef/issues/356 I am under impression that :from<native>:<version> is required in META6.json ... cc @niner

jonathanstowe commented 4 years ago

and needs to be be specified to the native trait

@jonathanstowe what is that?

Right. So if we have established for instance that both version 2 and version 3 of libgd work for the API used in the module then supplying

  sub gdImageGif(GD::Image, GD::File) is native('gd', v2) { ... };

Will not work on a system where v3 of the library is installed and

 sub gdImageGif(GD::Image, GD::File) is native('gd', v3) { ... };

Will not work on a system where only v2 is available. So something a bit more subtle needs to be done, that is a well understood problem and can be dealt with by selecting the available version of the library from a list of known working ones earlier. I have that in hand.

The meta thing is just a nice to have

after some discussion, especially that - ugexe/zef#356 I am under impression that :from<native>:<version> is required in META6.json ... cc @niner

I don't believe that is required no. I'm not sure how it could be, after all the installer doesn't even know that the module has used a native trait, and even if it could know that there are many different ways in which a module can determine and specify the library and version required. As I understand it, it is a hint as a convenience to the installer, and has no relation to the runtime specification of the library and version in the actual module,

niner commented 4 years ago

OpenSSL is in a similar place as it supports openssl 1.0 and 1.1. I've got a PR for solving it there: https://github.com/niner/openssl/blob/master/lib/OpenSSL/NativeLib.pm6#L37 and https://github.com/niner/openssl/blob/master/lib/OpenSSL/NativeLib.pm6#L25

For specifying multiple allowed versions in the META6.json: https://github.com/niner/openssl/blob/master/META6.json#L9

Note that I haven't submitted this PR yet because the support for "any" hasn't been merged into zef yet.

jonathanstowe commented 4 years ago

So just to summarise so this ticket has some actual deliverable, we are talking about two slightly orthogonal things: how the native library is specified in the module itself (which is required and bears upon whether the module will work at all ) and the way that the native dependency is specified in the META file (which is not required AFAIK and is just for the convenience of the installer and other tools.) So I am going to consider this ticket fixed when the first of those is remedied and the module will work with any of the supported versions of libgd installed. If the second is to be fixed then it should be done in a separate issue or PR so as not to have confusing conversations about two separate things at once.

jonathanstowe commented 4 years ago

OpenSSL is in a similar place as it supports openssl 1.0 and 1.1. I've got a PR for solving it there:

I'm probably going to go with something more like:

https://github.com/jonathanstowe/Crypt-SodiumPasswordHash/blob/master/lib/Crypt/SodiumPasswordHash.pm#L59-L79

Or some better abstraction thereof.

melezhik commented 4 years ago

I understand all that @jonathanstowe , thanks. My questions now more for @niner and GD is just a good case for such a questions:

if gd:from:<native> is required in META?
how do we specify both 2 and 3 versions in META?
based on https://github.com/ugexe/zef/issues/356 we established that META has to be used as a source of truth for installer to install external dependencies and to verify they are satisfied. Does it apply for GD module?

So far the information I've received from @jonathanstowe in this issue seems abit contradictory to the approach we've established before ...

Thanks

niner commented 4 years ago

I'm probably going to go with something more like: https://github.com/jonathanstowe/Crypt-SodiumPasswordHash/blob/master/lib/Cr ypt/SodiumPasswordHash.pm#L59-L79

As long as you don't use LibraryCheck. That one's using a Rakudo implementation detail...

niner commented 4 years ago

gd:from:<native> is required in META?

how do we specify both 2 and 3 versions in META?

As in the link to openssl I posted earlier: {"any": [{"name":"gd", "from": "native", "ver": "2"}, {"name":"gd", "from": "native", "ver": "3"}]},

melezhik commented 4 years ago

right. HOW does an installer choose which version to pick to install?

does it mean it had to sequentially try versions from any list and choose first available on an underlying system?

jonathanstowe commented 4 years ago

As far as I can tell any native dependencies in the META file are ignored by the most recent version of zef - if I put:

   "depends" : {
      "runtime" : {
         "requires" : [
            "LibraryCheck",
            "NativeHelpers::Array",
            "gd:from<native>",
            "zzzzz:from<native>"
         ]
      }
   },

It installs fine without any complaint about the non-existent zzzzz

niner commented 4 years ago

On Montag, 29. Juni 2020 16:18:12 CEST Jonathan Stowe wrote:

As far as I can tell any native dependencies in the META file are ignored by the most recent version of zef - if I put:

That's what we're trying to change :)

melezhik commented 4 years ago

@jonathanstowe I think will zef will fail if library declared in from:native is not installed , however zef won't try to install a dependency. But Sparrow will - https://sparrowdo.wordpress.com/2020/06/22/install-raku-modules-and-external-dependencies-using-sparrow/

jonathanstowe commented 4 years ago

As long as you don't use LibraryCheck. That one's using a Rakudo implementation detail...

Well you see it wasn't until someone sent a PR because they didn't like that it was doing an EVAL.

niner commented 4 years ago

right. HOW does installer choose which version to pick to install?

Try the listed alternatives in order, install the first available.

melezhik commented 4 years ago

@niner I see. thanks. so see my first question. gd:from:native is required in META, is it?

jonathanstowe commented 4 years ago

I think zef will fail if library declared in from:native is not installed ,

No it doesn't. If a completely bogus native library is declared it will still install fine.

niner commented 4 years ago

Well you see it wasn't until someone sent a PR because they didn't like that it was doing an EVAL.

Anyway, it clearly shows that NativeCall is missing a documented way to check whether a library can be loaded. In hindsight, linking setup of a native callsite and loading a library that closely was a mistake.

niner commented 4 years ago

On Montag, 29. Juni 2020 16:25:13 CEST Alexey Melezhik wrote:

@niner I see. thanks. so see my first question. gd:from:native is required in META, is it?

No dependency information is really required as such. But the information does help the user. By providing dependency information a module author makes it easier for the user to install and use modules. At the very least, users know what to install and of course that can be automated for them.

Information about native dependencies in particular will open the possibilities to:

have those native dependencies automatically checked by e.g. zef
have those native dependencies automatically installed
automatically package the module for Linux distributations
get CI support automatically (as installation is completely automated)
have the module automatically tested against every change to rakudo, i.e. prevent rakudo developers from breaking your module

jonathanstowe commented 4 years ago

As I said before the dependencies can't be required, because the installer itself doesn't know whether there are any to be be required, and if the requirement was on the installer to check in the package in question whether there were any dependencies that needed to be declared, then the point of them being declared at all becomes moot because the installer just found them anyway to check whether they were required or not.

melezhik commented 4 years ago

@niner so, from:native is only required if a module author wants to declare an interface to automate external dependencies installation, but not required for module itself, right?

melezhik commented 4 years ago

@niner could you please provide a patch for META6.json file so I could see your understanding of native dependecies declaration? thanks

raku-community-modules / GD

The module is not installable on CentOS #12