search

ugexe / zef

Raku Module Management
Artistic License 2.0
207 stars 44 forks source link

can't install .tar.gz files w/ META.info #492

Closed masukomi closed 1 year ago

masukomi commented 1 year ago

Time::Duration:ver<2.00> is currently uninstallable with the zef 0.14.6 and rakudo v2022.12 under macOS Ventura 13.1 when pulled from online, however if you tell zef to install the manually decompressed contents of the archive it pulled down, it installs fine.

This appears to be an issue with the tar extraction.

Note that Time::Duration is not itself the problem. It's just a random one that uses tar.gz and has a META.info instead of a META.json

Context

Expected Behavior

zef install "Time::Duration:ver<2.00>" succeeds

Actual Behavior

zef install "Time::Duration:ver<2.00>" complains about missing META6.json file and fails to notice the META.info file.

Steps to Reproduce

  1. macos on Ventura 13.1
  2. rakudo v2022.12
  3. run zef install "Time::Duration:ver<2.00>"
❯ zef --debug install "Time::Duration:ver<2.00>"
===> Searching for: Time::Duration:ver<2.00>
===> Found: Time::Duration:ver<2.00> [via Zef::Repository::Ecosystems<rea>]
===> Fetching: Time::Duration:ver<2.00>
[Time::Duration] Fetching https://raw.githubusercontent.com/raku/REA/main/archive/T/Time%3A%3ADuration/Time%3A%3ADuration%3Aver%3C2.00%3E%3Aauth%3Cgithub%3Adagurval%3E.tar.gz with plugin: Zef::Service::Shell::curl+{<anon|1>}
===> Fetching [OK]: Time::Duration:ver<2.00> to /var/folders/x6/fq4x1sys07v5ngrc4p893k6w0000gn/T//.zef/1673993479.8366.3267.863582444396/Time%3A%3ADuration%3Aver%3C2.00%3E%3Aauth%3Cgithub%3Adagurval%3E.tar.gz
===> Extracting: Time::Duration:ver<2.00>
===> Extraction: Failed to find a META6.json file for Time::Duration:ver<2.00> -- failure is likely
[Time::Duration] Extracting with plugin: Zef::Service::Shell::tar+{<anon|1>}
===> Extraction [OK]: Time::Duration:ver<2.00> to /var/folders/x6/fq4x1sys07v5ngrc4p893k6w0000gn/T//.zef/Time%3A%3ADuration%3Aver%3C2.00%3E%3Aauth%3Cgithub%3Adagurval%3E.tar.gz
Type check failed for return value; expected IO::Path but got Any (Any)

Clue after poking around in the source

$extracted-to.IO.add('META.info')

from line 620 in Client.rakumod if !$meta6-prefix and my $meta-info = $extracted-to.IO.add('META.info') and $meta-info.e

is 

IO::Path.new("/var/folders/x6/fq4x1sys07v5ngrc4p893k6w0000gn/T//.zef/Time\%3A\%3ADuration\%3Aver\%3C2.00\%3E\%3Aauth\%3Cgithub\%3Adagurval\%3E.tar.gz/META.info", :SPEC(IO::Spec::Unix), :CWD("/Users/masukomi/workspace/reference/raku/zef"))

Note that it's .tar.gz/META.info When i work with the downloaded data and run tar -xzvf on the .tar.gz file it produces a folder named p6-Time-Duration-master which DOES contain the META.info

It seems that the IO::Path being reuturned by the extract method in Tar.rakumod is incorrect as it seems like it should be ending with p6-Time-Duration-master

within Tar.rakumod's extract method the $archive-file passed in was 

/var/folders/x6/fq4x1sys07v5ngrc4p893k6w0000gn/T//.zef/1673993437.7474.8411.245522545481/Time\%3A\%3ADuration\%3Aver\%3C2.00\%3E\%3Aauth\%3Cgithub\%3Adagurval\%3E.tar.gz

and the resulting $extract-to was 

/var/folders/x6/fq4x1sys07v5ngrc4p893k6w0000gn/T//.zef/Time\%3A\%3ADuration\%3Aver\%3C2.00\%3E\%3Aauth\%3Cgithub\%3Adagurval\%3E.tar.gz

Your Environment

raku -v

❯ raku -v
Welcome to Rakudo™ v2022.12.
Implementing the Raku® Programming Language v6.d.
Built on MoarVM version 2022.12.

zef list --installed

❯ zef list --installed
===> Found via inst#/Users/masukomi/.rakubrew/versions/moar-2022.12/share/perl6/site
Base64:ver<0.1.0>:auth<github:ugexe>
CBOR::Simple:ver<0.1.3>:auth<zef:japhb>
Cro::Core:ver<0.8.7>
Cro::Core:ver<0.8.9>:auth<zef:cro>
Cro::HTTP:ver<0.8.9>:auth<zef:cro>
Cro::TLS:ver<0.8.7>
Cro::TLS:ver<0.8.9>:auth<zef:cro>
Cro::WebSocket:ver<0.8.9>:auth<zef:cro>
Crypt::Random:ver<0.4.1>:auth<github:skinkade>
DateTime::Parse:ver<0.9.3>:auth<github:sergot>
Digest::HMAC:ver<1.0.5>:auth<zef:jjmerelo>
Digest::SHA1::Native:ver<0.05>
Digest:ver<0.28.1>:auth<zef:grondilu>
Docker::File:ver<1.0>:auth<github:jnthn>
File::Find:ver<0.1.1>
File::Ignore:ver<1.1>:auth<Jonathan Worthington <jnthn@jnthn.net>>
File::Which:ver<1.0.4>
HTML::Escape:ver<0.0.1>
HTTP::HPACK:ver<1.0.0>:auth<zef:jnthn>
IO::Path::ChildSecure:ver<1.2>:auth<zef:raku-community-modules>
IO::Socket::Async::SSL:ver<0.7.14>:auth<zef:jnthn>
JSON::Class:ver<0.0.19>:auth<zef:jonathanstowe>:api<1.0>
JSON::Fast:ver<0.17>:auth<cpan:TIMOTIMO>
JSON::JWT:ver<1.1.1>:auth<zef:raku-community-modules>
JSON::Marshal:ver<0.0.24>:auth<zef:jonathanstowe>:api<1.0>
JSON::Name:ver<0.0.7>:auth<zef:jonathanstowe>:api<1.0>
JSON::OptIn:ver<0.0.2>:auth<zef:jonathanstowe>
JSON::Unmarshal:ver<0.11>:auth<zef:raku-community-modules>
LibraryMake:ver<1.0.0>:auth<github:retupmoca>
Linenoise:ver<0.1.2>:auth<zef:raku-community-modules>
Log::Timeline:ver<0.5.1>:auth<zef:jnthn>
META6:ver<0.0.29>:auth<zef:jonathanstowe>:api<1.0>
MIME::Base64:ver<1.2.3>:auth<zef:raku-community-modules>
OO::Monitors:ver<1.1.1>
OpenSSL:ver<0.2.0>:auth<Filip Sergot>
PathTools:ver<0.2.0>:auth<github:ugexe>
Shell::Command
Terminal::ANSIColor:ver<0.9>:auth<zef:lizmat>
TinyFloats:ver<0.0.4>:auth<zef:japhb>
YAMLish:ver<0.0.6>
cro:ver<0.8.9>:auth<zef:cro>
fez:ver<39>:auth<zef:tony-o>:api<0>
if:ver<0.1.1>:auth<github:FROGGS>
zef:ver<0.14.6>:auth<github:ugexe>:api<0>
===> Found via inst#/Users/masukomi/.rakubrew/versions/moar-2022.12/share/perl6/core
rakudo:ver<2022.12>:auth<Yet Another Society>
ugexe commented 1 year ago

To preface things, META.info support has not been supported in rakudo for years. The only reason anything still using a META.info would have worked in rakudo is due to zef doing that hacky workaround of creating an actual META6.json file (which rakudo could then understand). Indeed there is a 4 year old comment in zef to remove the piece of code responsible for generating that file.

At this point I'm not inclined to fix anything related to META.info. Authors have had ample time to update their distributions, and if it hasn't happened yet its probably safe to assume the distribution itself is on its way to bitrot. Since it has presumably been broken in zef for awhile, and you're the first person I've seen run into this issue, I'm inclined to update zef to abort entirely and treat the distribution as invalid. In fact it might be worth removing those distributions from the REA ecosystem altogether unless someone e.g. moves them to raku-community-modules so the META6.json file can be added + a new release.

melezhik commented 1 year ago

Just my two cents here - also reproducible on Debian / Rakudo 2021.12 , with shipped zef - https://ci.sparrowhub.io/report/2645

masukomi commented 1 year ago

To preface things, META.info support has not been supported in rakudo for years.

2 thoughts:

  1. I think the problem is more significant than META.info support. The problem, as far as I can tell is that Tar.rakumod seems to be either not extracting, or not returning the correct folder path when it does. I assume this would effect anything that needs Tar extraction.
  2. if META.info is support is going to be abandoned (seems reasonable) it would probably be best to simply remove the META.info related code paths so that we don't have future people wasting time trying to debug things that won't be fixed.
melezhik commented 1 year ago

@ugexe @masukomi interesting ... the same build succeeds if using pakku instead of zef - https://ci.sparrowhub.io/report/2661

melezhik commented 1 year ago

btw AFIK pakku uses libarchive to do TAR/GZ manipulations, I am not sure what zef does, any maybe with some distros this might be a problem ... 🤔

ugexe commented 1 year ago

It's not really interesting in that we know there is a bug related directly to working around the deprecation of META.info, and that I'm not inclined to fix it to continue working. It was never in question that the extraction utility (i.e. tar) was doing the wrong thing.

In all reality things with only a META.info shouldn't be install-able period, as zef has warned about failure being likely in this exact scenario for 5 years. It just so happens this workaround has finally bitrotted after all these years, which also happens to be a good opportunity to recognize that we are explicitly dropping support altogether.