search

ugexe / zef

Raku Module Management
Artistic License 2.0
206 stars 44 forks source link

zef can't find any distribution that has ":ver<*>" in its name #553

Closed paultcochrane closed 4 months ago

paultcochrane commented 4 months ago

Context

While installing Selenium::WebDriver I found its tests to be failing. After some digging, I found that one issue causing the failures was because one of its dependencies File::Zip was missing the unzip method:

t/05-firefox.rakutest ......
1..91
ok 1 - 'use Selenium::WebDriver::Firefox' worked!
unzipping /<snip>/raku-selenium-webdriver/resources/firefox_extension/webdriver.xpi into /tmp/_g9TqyCQk1/perl6-selenium-webdriver/extensions/fxdriver@googlecode.com
No such method 'unzip' for invocant of type 'File::Zip'
  in method start at /<snip>/raku-selenium-webdriver/lib/Selenium/WebDriver/Firefox.rakumod (Selenium::WebDriver::Firefox) line 59
  in submethod BUILD at /<snip>/raku-selenium-webdriver/lib/Selenium/WebDriver/Wire.rakumod (Selenium::WebDriver::Wire) line 47
  in block <unit> at t/05-firefox.rakutest line 45

After more digging, I found that Selenium::WebDriver expects one https://raku.land/github:azawawi/File::Zip, however zef finds another: https://raku.land/cpan:TYIL/File::Zip. This second version of File::Zip doesn't implement the same API and hence the unzip method isn't found. It's unfortunate that Selenium::WebDriver just expects File::Zip to be used (without any further restriction as to which File::Zip to use), however this is a separate issue.

When searching for File::Zip in zef we get this output:

zef search File::Zip
===> Found 4 results
--------------------------------------------------------------------------------------------------------------------------------------
ID|From                            |Package                                      |Description
--------------------------------------------------------------------------------------------------------------------------------------
0 |Zef::Repository::Ecosystems<rea>|File::Zip                                    |Perl 6 API to the ZIP file format
1 |Zef::Repository::Ecosystems<rea>|File::Zip:ver<0.1.2>:auth<cpan:TYIL>:api<0>  |A wrapper around zip files, using zip/unzip commands
2 |Zef::Repository::Ecosystems<rea>|File::Zip:ver<0.1.1>:auth<cpan:TYIL>:api<0>  |A wrapper around zip files, using zip/unzip commands
3 |Zef::Repository::LocalCache     |File::Zip:ver<0.1.2>:auth<gitlab:tyil>:api<0>|A wrapper around zip files, using zip/unzip commands
--------------------------------------------------------------------------------------------------------------------------------------

Note that the package name for ID 0 doesn't include any metadata. One guess for why this might be is because https://raku.land/github:azawawi/File::Zip uses :ver<*> as its version info and that zef is somehow not handling that correctly. One potential solution to this issue would be to patch the offending File::Zip dist to use semantic versioning, however since the dist hasn't been updated since 2017, this change might take a while to land.

Another solution to the issue would be for the user to use

zef -v install https://github.com/azawawi/perl6-file-zip.git

however that's definitely LTA for users, especially if one is used to the zef install Package::Name invocation (which is also what is documented in e.g. Selenium::WebDriver) and because when zef tries to install File::Zip as part of installation of another package, it won't use the GitHub link and will just use the name instead and hence the "incorrect" (for Selenium::WebDriver at least) will still be used.

I also tried specifying the :auth<> information as a way to directly specify which dist was required, however this didn't work as hoped:

$ zef install 'File::Zip:auth<github:azawawi>'
===> Searching for: File::Zip:auth<github:azawawi>
No candidates found matching identity: File::Zip:auth<github:azawawi>
$ zef install 'File::Zip:auth<zef:azawawi>'
===> Searching for: File::Zip:auth<zef:azawawi>
No candidates found matching identity: File::Zip:auth<zef:azawawi>

The duplication of File::Zip dists has been noticed before: https://github.com/azawawi/raku-selenium-webdriver/issues/27. When asking about the duplication and how to resolve the issue on the #raku irc channel, gfldex mentioned that this seemed to be a bug in zef, hence why I'm bringing up this issue here.

Expected Behavior

It's hard to know exactly what the expected behaviour should be in this case because there's a naming conflict between the two dists and zef will have to make a sensible choice about which one to choose if the user simply tries to just install File::Zip. In the current situation, since https://raku.land/cpan:TYIL/File::Zip has a newer release date, it would make sense to prefer installation of that dist. However, what would be nice to be able to do would be something like:

$ zef install 'File::Zip:auth<github:azawawi>'

and then https://raku.land/github:azawawi/File::Zip would be installed instead of https://raku.land/cpan:TYIL/File::Zip, which would be the preferred behaviour in the current case (e.g. installing Selenium::WebDriver).

Actual Behavior

https://raku.land/cpan:TYIL/File::Zip is installed instead of https://raku.land/github:azawawi/File::Zip.

Steps to Reproduce

zef install --debug File::Zip
===> Searching for: File::Zip
===> Found: File::Zip:ver<0.1.2>:auth<cpan:TYIL>:api<0> [via Zef::Repository::Ecosystems<rea>]
===> Dependencies: File::Temp, File::Which
===> Fetching: File::Zip
[File::Zip] Fetching https://raw.githubusercontent.com/raku/REA/main/archive/F/File%3A%3AZip/File%3A%3AZip%3Aver%3C0.1.2%3E%3Aauth%3Ccpan%3ATYIL%3E.tar.gz with plugin: Zef::Service::Shell::curl
===> Fetching [OK]: File::Zip:ver<0.1.2>:auth<cpan:TYIL>:api<0> to /tmp/.zef/1710842307.3088775/1710842326.3088775.6601.189851172996/File%3A%3AZip%3Aver%3C0.1.2%3E%3Aauth%3Ccpan%3ATYIL%3E.tar.gz
===> Extracting: File::Zip
[File::Zip] Extracting with plugin: Zef::Service::Shell::tar
===> Extraction [OK]: File::Zip to /tmp/.zef/1710842307.3088775/File%3A%3AZip%3Aver%3C0.1.2%3E%3Aauth%3Ccpan%3ATYIL%3E.tar.gz
===> Filtering: File::Zip:ver<0.1.2>:auth<gitlab:tyil>:api<0>
===> Filtering [OK] for File::Zip:ver<0.1.2>:auth<gitlab:tyil>:api<0>
===> # SKIP: No need to build File::Zip:ver<0.1.2>:auth<gitlab:tyil>:api<0>
===> Staging File::Zip:ver<0.1.2>:auth<gitlab:tyil>:api<0>
===> Staging [OK] for File::Zip:ver<0.1.2>:auth<gitlab:tyil>:api<0>
===> Testing: File::Zip:ver<0.1.2>:auth<gitlab:tyil>:api<0>
[File::Zip] Testing with plugin: Zef::Service::TAP
[File::Zip] 1..3
[File::Zip] # Subtest: Instantiate existing zip
[File::Zip]     1..4
[File::Zip]     ok 1 - Path to zip file is accessible
[File::Zip]     ok 2 - Zip file exists
[File::Zip]     # Subtest: List files in zip
[File::Zip]         1..3
[File::Zip]         ok 1 - Correct number of files found in zip
[File::Zip]         ok 2 - Expected file found
[File::Zip]         ok 3 - Length is correct
[File::Zip]     ok 3 - List files in zip
[File::Zip]     # Subtest: Extract files
[File::Zip]         1..2
[File::Zip]         ok 1 - .extract reports success
[File::Zip]         ok 2 - File was correctly extracted
[File::Zip]     ok 4 - Extract files
[File::Zip] ok 1 - Instantiate existing zip
[File::Zip] # Subtest: Instantiate existing, non-zip file
[File::Zip]     1..1
[File::Zip]     ok 1 - Dies when trying to load a file which is not a zip
[File::Zip] ok 2 - Instantiate existing, non-zip file
[File::Zip] # Subtest: Instantiate non-existing zip
[File::Zip]     1..1
[File::Zip]     ok 1 - Dies when trying to load a non-existing file
[File::Zip] ok 3 - Instantiate non-existing zip
[File::Zip] t/01-file-zip.t .. ok
[File::Zip] All tests successful.
[File::Zip] Files=1, Tests=3,  1 wallclock secs
[File::Zip] Result: PASS
===> Testing [OK] for File::Zip:ver<0.1.2>:auth<gitlab:tyil>:api<0>
===> Installing staged code
===> Installing: File::Zip:ver<0.1.2>:auth<gitlab:tyil>:api<0>
===> Install [OK] for File::Zip:ver<0.1.2>:auth<gitlab:tyil>:api<0>
===> Installation [OK]

Your Environment

Welcome to Rakudo™ v2024.02.
Implementing the Raku® Programming Language v6.d.
Built on MoarVM version 2024.02.
zef list --installed
===> Found via inst#/home/cochrane/.rakubrew/versions/moar-2024.02/share/perl6/site
App::Mi6:ver<3.0.4>:auth<zef:skaji>
App::Prove6:ver<0.0.17>:auth<cpan:LEONT>
Compress::Zlib::Raw:ver<1.0.1>:auth<github:retupmoca>
Compress::Zlib:ver<1.1.0>:auth<github:retupmoca>
DateTime::Parse:ver<0.9.3>:auth<github:sergot>
Encode:ver<0.0.4>:auth<github:sergot>
File::Directory::Tree:ver<0.1>:auth<zef:labster>
File::Find:ver<0.2.1>:auth<zef:raku-community-modules>
File::Temp:ver<0.0.10>:auth<zef:rbt>
File::Which:ver<1.0.4>
File::Zip:ver<0.1.2>:auth<gitlab:tyil>:api<0>
Getopt::Long:ver<0.4.2>
HTTP::Status:ver<0.0.4>:auth<zef:lizmat>
HTTP::Tiny:ver<0.2.5>:auth<zef:jjatria>
HTTP::UserAgent:ver<1.1.52>:auth<github:sergot>
IO::Capture::Simple:ver<v.0.0.2>:auth<zef:jjmerelo>
IO::Socket::SSL:ver<0.0.3>:auth<github:sergot>
JSON::Class:ver<0.0.21>:auth<zef:jonathanstowe>:api<1.0>
JSON::Fast:ver<0.19>:auth<cpan:TIMOTIMO>
JSON::Marshal:ver<0.0.25>:auth<zef:jonathanstowe>:api<1.0>
JSON::Name:ver<0.0.7>:auth<zef:jonathanstowe>:api<1.0>
JSON::OptIn:ver<0.0.2>:auth<zef:jonathanstowe>
JSON::Tiny:ver<1.0>
JSON::Unmarshal:ver<0.15>:auth<zef:raku-community-modules>
LibraryMake:ver<1.0.5>:auth<zef:jjmerelo>
License::SPDX:ver<3.23.0>:auth<zef:jonathanstowe>:api<1.0>
Linenoise:ver<0.1.2>:auth<zef:raku-community-modules>
META6:ver<0.0.30>:auth<zef:jonathanstowe>:api<1.0>
MIME::Base64:ver<1.2.3>:auth<zef:raku-community-modules>
OO::Monitors:ver<1.1.1>
OpenSSL:ver<0.2.2>:auth<github:sergot>
Path::Finder:ver<0.4.7>:auth<zef:leont>
PathTools:ver<0.2.0>:auth<github:ugexe>
Pod::Load:ver<0.7.2>:auth<zef:jjmerelo>
Pod::To::HTML:ver<0.8.1>:auth<github:Raku>
Pod::To::Markdown:ver<0.2.1>:auth<github:softmoth>
Pod::Usage:ver<0.0.1>:auth<zef:leont>
Selenium::WebDriver:ver<0.0.2>
Shell::Command:ver<1.1>:auth<zef:raku-community-modules>
TAP:ver<0.3.14>
Template6:ver<0.14.0>:auth<zef:raku-community-modules>
Template::Mustache:ver<1.2.3>:auth<github:softmoth>
Test::META:ver<0.0.20>:auth<zef:jonathanstowe>:api<1.0>
Test::Output:ver<1.001006>:auth<zef:raku-community-modules>
Test::Util::ServerPort:ver<0.0.5>:auth<zef:jonathanstowe>:api<1.0>
URI::Encode:ver<0.09>:auth<zef:raku-community-modules>
URI:ver<0.3.7>:auth<zef:raku-community-modules>
fez:ver<55>:auth<zef:tony-o>:api<0>
sigpipe:ver<0.0.3>:auth<zef:leont>
zef:ver<0.21.4>:auth<github:ugexe>:api<0>
===> Found via inst#/home/cochrane/.rakubrew/versions/moar-2024.02/share/perl6/core
rakudo:ver<2024.02>:auth<Yet Another Society>

Wrapping up

I think that's all of the information I have about the issue at present. If you need more information or feedback from me, please let me know :-)

ugexe commented 4 months ago

I think this is the correct behavior.

One guess for why this might be is because https://raku.land/github:azawawi/File::Zip uses :ver<*> as its version info and that zef is somehow not handling that correctly

use Foo:ver<*> is the same as use Foo, and hence zef shows Foo to be less verbose. However when a distribution declares their version as * (which is generally frowned upon and will probably be disallowed entirely in the future) it gets treated as 0. If it wasn't treated as version 0 and indeed like a version * then it would always match even when a user requests an exact version: 

$ raku -e 'say Version.new("1") ~~ Version.new("*")'
True
$ raku -e 'say Version.new("2") ~~ Version.new("*")'
True

so when sorting distributions to pick the "highest" version * will always be last.

However, what would be nice to be able to do would be something like:

$ zef install 'File::Zip:auth'

You can do this for distributions that declare their auth. The example shown is a bit confusing because RakuLand and REA are only inferring it as having an auth of github:azawawi, but really it has no auth declared at all (see: https://github.com/azawawi/perl6-file-zip/blob/master/META6.json) and thus File::Zip:auth<github:azawawi> won't match anything (similar to if you tried to use File::Zip:auth<github:azawawi> after installing it).

paultcochrane commented 4 months ago

Thanks for the quick answer! Also thanks for the tip about the * in :ver<*> being treated as 0, because it turns out that this invocation installs the desired dist:

$ zef install 'File::Zip:ver<0>' --debug
===> Searching for: File::Zip:ver<0>
===> Found: File::Zip [via Zef::Repository::Ecosystems<rea>]
===> Dependencies: Compress::Zlib
===> Fetching: File::Zip:ver<0>
[File::Zip] Fetching https://raw.githubusercontent.com/raku/REA/main/archive/F/File%3A%3AZip/File%3A%3AZip%3Aver%3C%2A%3E%3Aauth%3Cgithub%3Aazawawi%3E.tar.gz with plugin: Zef::Service::Shell::curl
===> Fetching [OK]: File::Zip to /tmp/.zef/1710858361.3997782/1710858379.3997782.3180.8889149395304/File%3A%3AZip%3Aver%3C%2A%3E%3Aauth%3Cgithub%3Aazawawi%3E.tar.gz
===> Extracting: File::Zip:ver<0>
[File::Zip] Extracting with plugin: Zef::Service::Shell::tar
===> Extraction [OK]: File::Zip:ver<0> to /tmp/.zef/1710858361.3997782/File%3A%3AZip%3Aver%3C%2A%3E%3Aauth%3Cgithub%3Aazawawi%3E.tar.gz
===> Filtering: File::Zip
===> Filtering [OK] for File::Zip
===> # SKIP: No need to build File::Zip
===> Staging File::Zip
===> Staging [OK] for File::Zip
===> Testing: File::Zip
[File::Zip] Testing with plugin: Zef::Service::TAP
[File::Zip] 1..1
[File::Zip] ok 1 - 'use File::Zip' worked!
[File::Zip] t/01-load.t .. ok
[File::Zip] All tests successful.
[File::Zip] Files=1, Tests=1,  3 wallclock secs
[File::Zip] Result: PASS
===> Testing [OK] for File::Zip
===> Installing staged code
===> Installing: File::Zip
===> Install [OK] for File::Zip
===> Installation [OK]

I'm pretty sure I wouldn't have stumbled across the idea of using :ver<0> without any help :-)

I've also submitted a PR to https://github.com/azawawi/perl6-file-zip to add naming authority metadata to the dist so that modules and code downstream from the dist can more easily specify that they want this particular version of File::Zip.

I now consider this issue to be resolved. Thanks for your help!