On .install, check a distro's /resources file paths are identical to those of the META6.json file's <resources>

tbrowder commented 4 days ago

This should be a fundamental check due to the close ties between Raku distributions and their 'META6.json' file. Unless those two "resources" file path lists match, it is impossible to download a '/resources' directory's file payload without foreknowledge of those paths.

But, with a match, the distribution author can easily provide the means to do so.

Currently, and apparently, such is not done since I have never seen such an error using 'zef install . --debug' with known mismatches.

Context

Among other things, current Raku documentation does not clearly point out how critical that a "resources" match is to downloading '/resources' from an installed distribution.

Expected Behavior

In a distribution repository directory with no listed 'resources' under '/resources' or in the 'META6.json file, no change to current bevavior is needed.

But for any other case, report any mismatch:

zef test . --debug
WARNING: /resources file paths do not match those of META6.json
   /resources has files:
      ...file path list...
   META6.json.resources:
      ...file path list...

The "WARNING" should also appear without the '--debug' option.

Upon 'zef install Some::Module', throw an error and describe the mismatch. Also add a message that the user should file an issue with that distribution's author.

Actual Behavior:

No warning is given, even with the '--debug' option to 'zef'.

Steps to Reproduce

# Clone my module 'Foo::Bar' into the local directory.
# That module has intentionally mismatched "resources" file paths.
$ git clone https://github.com/tbrowder/Foo-Bar
$ zef test . --debug
===> Testing: Foo::Bar:ver<0.0.1>:auth<zef:tbrowder>
[Foo::Bar] Testing with plugin: Zef::Service::TAP
[Foo::Bar] 1..1
[Foo::Bar] ok 1 - Module Foo::Bar used okay
[Foo::Bar] t/0-load.rakutest .. ok
[Foo::Bar] All tests successful.
[Foo::Bar] Files=1, Tests=1,  0 wallclock secs
[Foo::Bar] Result: PASS
===> Testing [OK] for Foo::Bar:ver<0.0.1>:auth<zef:tbrowder>

Your Environment

raku -v

Welcome to Rakudo™ v2024.09.
Implementing the Raku® Programming Language v6.d.
Built on MoarVM version 2024.09.

zef list --installed

zef:ver<0.22.1>:auth<github:ugexe>:api<0>
[and MANY others]

ugexe commented 4 days ago

Indeed you can already know if there is a file listed in META6.json that doesn't exist by running zef install . since rakudo itself will raise an exception. However when it comes to making noise when files exist on the file system but aren't listed in the META6.json I'm inclined to disagree. It is perfectly reasonably to have files in various directories of a repository you also use as a raku distribution, and there is no reason that should warn for anyone but a module author (since a general user shouldn't care, and if they did they'd really be interested in all authoring related tests).

tbrowder commented 4 days ago

Nick, I respectfully disagree. The example I showed (Foo::Bar) is a distro I published last week (I didn't see it mentioned in the Raku Weekly).

Clone it, cd to it, visually confirm the "resources" mismatch, and run

zef install .

and I see nothing about a "resources" mismatch.

ugexe commented 3 days ago

Clone it, cd to it, visually confirm the "resources" mismatch, and run

Please re-read what I said and compare it to your repository:

Indeed you can already know if there is a file listed in META6.json that doesn't exist by running zef install . since rakudo itself will raise an exception.

In your repo the resources field is empty, and thus it does not contain a file that does not exist. What you have is files in a directory that are not listed on the META6.json, which I argue:

It is perfectly reasonably to have files in various directories of a repository you also use as a raku distribution

Specifically it is expected that a user can have files anywhere in their repository (except Build.rakumod) and it doesn't matter as long as the META6.json has the necessary stuff listed.

ugexe commented 3 days ago

While I don't think it makes sense for such a test to be run for anyone other than module authors, I've gone ahead and wrote a basic cut and paste-able test that module authors can use to test their distribution before authoring changes: https://gist.github.com/ugexe/c32f3a8128237c30a291f9eefb03e520

Remember that this is indeed an authoring issue: the only person who can act on this information not being correct is the author of the distribution. Therefore ideally only the distribution author is running these checks, not users installing stuff via zef

tbrowder commented 3 days ago

That's fine, thanks.

But what about an alternative for authors? Consider this:

The author is beavering away in his or her repo and wants to check things and runs:

zef lint .

Output:

Mismatch between /resources and META6.json.
/resources:
    ...
META6.json.resources:
    ...

ugexe / zef