Does Pakku avoid double precompilation?

[AlexDaniel]

So my understanding is that zef will normally precompile the module in the Testing phase and then once again when the module is installed. I haven't done any measurements but I'm guessing that this roughly doubles the time you'd have to wait in an ideal scenario. I'm very interested in seeing this improved because it can significantly reduce the time Blin takes to run.

[hythm7]

@AlexDaniel My understanding is that Rakudo does the precompilation. for example when Pakku test a module Rakudo will need to precompile it to load the module, also when installing the module Rakudo will precompile it after installing the module to its CompUnit::Repository::Installation. I can't think of a way to avoid precompiling the module during the testing phase.

That being said, I have done a small measurement buy running Blin with around 1000 modules, test-modules. and here are the timings for both Pakku and Zef:

The measurement was done on laptop with 4 CPUs and 8GB Memory:

Pakku:

hythm@galaxy$ time PERL6LIB=lib bin/blin.p6 $(cat test-modules)
...
...
🥞 Saving results
🥞🥞 Saving the overview
🥞🥞 Saving the failure output
🥞🥞 Saving the json output
🥞🥞 Saving the dot file
🥞🥞 Creating SVG/PNG images from the dot file
🥞 Cleaning up

real    165m57.902s
user    613m34.353s
sys 26m57.478s

Zef:

hythm@galaxy$ time PERL6LIB=lib bin/blin.p6 $(cat test-modules)
...
...
⏳ 299 out of 1069 modules processed
⏳ 299 out of 1069 modules processed
🥞🥞🥞 Testing Foo (new)
🥞🥞🥞 Testing Grammar::HTTP (new)
🥞🥞🥞 Testing Event::Emitter for flappiness
^C

real    208m12.593s
user    774m8.707s
sys 24m47.402s

# I canceled `Zef` run because I noticed it took longer time and it was almost 300 modules and remaining around 700.

So unless I'm doing something terribly wrong here, I think the difference in timings is because Zef loads the entire ecosystem with every run, while Pakku version pupa gets the distributions meta from online Recommendation Manager Pakku::RecMan and doesn't not need to load the ecosystem first.

The reason I selected 1000 modules only and not all available modules is because of this not yet resolved Pakku issue. Since Blin installs modules into a custom repository (not one of default Raku repositories like home site vendor), I needed to select modules that will not fail the testing phase due to the mentioned Pakku issue.

Below is a diff of the changes I made for Blin to use Pakku: Pakku binary is installed to /home/hythm/.pakku/bin/pakku on my machine.

diff --git a/lib/Blin/Processing.pm6 b/lib/Blin/Processing.pm6
index f38c63a..58e4dc7 100644
--- a/lib/Blin/Processing.pm6
+++ b/lib/Blin/Processing.pm6
@@ -396,17 +396,16 @@ sub test-module($full-commit-hash, $module,
         } else { # normal module
             $result = get-output $binary-path,
                       ‘--’,
-                      $zef-path.add(‘/bin/zef’),
-                      “--config-path=$zef-config-path”,
-                      <--verbose --force-build --force-install>,
-                      ($testable ?? ‘--force-test’ !! ‘--/test’),
-                      <--/depends --/test-depends --/build-depends>,
-                      ‘install’,
-                      “--to=inst#$install-path”, $module.name,
+                      '/home/hythm/.pakku/bin/pakku',
+                      'add',
+                      'force',
+                      ($testable ?? ‘test’ !! ‘notest’),
+                      'nodeps',
+                      “to $install-path”, $module.name,
                       :stdin(‘’), :$timeout, ENV => %tweaked-env, :!chomp;
         }
         # XXX ↓ this workaround looks stupid
-        $result<exit-code> = 1 if $result<output>.contains: ‘[FAIL]:’;
+        $result<exit-code> = 1 if $result<output>.contains: ‘Nofun’;
         $result
     }

lines 1-29/29 (END)

[AlexDaniel]

@hythm7 wow, I almost missed this among the notification spam. Absolutely fantastic! I did know that there is some overhead from zef, but I didn't expect at all that it'd be that much.

I can't think of a way to avoid precompiling the module during the testing phase.

Yeah, I remember some talk and ideas about just taking the files and moving them elsewhere. It's probably much harder than it sounds.

However… in case of Blin, can't you use the same install directory for testing? Blin doesn't mind. Maybe a command line switch, or something? I dunno, does this idea even make sense?

The reason I selected 1000 modules only and not all available modules is because of this not yet resolved Pakku issue

I'd be happy to see a PR to Blin to make it be able to use Pakku once it can test all modules. Or just ping me once the issue is fixed. :) Probably needs to be done in a way so that it's possible to switch between package managers, but if it is really that fast I'd be happy to make it use Pakku by default.

[hythm7]

However… in case of Blin, can't you use the same install directory for testing? Blin doesn't mind. Maybe a command line switch, or something? I dunno, does this idea even make sense?

This can work, I guess, one need to install the module first (without testing) to /some/dir then would include or use inst#/some/dir and test the module. but I think this logic should exist in Blin rather than in the package manager.

I'd be happy to see a PR to Blin to make it be able to use Pakku once it can test all modules. Or just ping me once the issue is fixed. :) Probably needs to be done in a way so that it's possible to switch between package managers

Pakku can test all modules now. I also made these changes to support switching between package managers in Blin, like bin/blin.p6 --pm=zef or bin/blin.p6 --pm=pakku, let me know if any changes needed and when all good I will submit PR.

on a side note: I have noticed that some modules test successfully when run by Pakku and Zef directly, but fail the first time only when tested by Blin, then when Blin test the module for flappiness it always succeeds. (cro and Cro::WebSocket show this behavior when tested by Blin).

[AlexDaniel]

but I think this logic should exist in Blin rather than in the package manager

Hm. Good point. That said, I'm always worried that if modules are installed in a different manner that they will show/hide different bugs. I think there were weird cases like this, but maybe it will be fine.

I have noticed that some modules test successfully when run by Pakku and Zef directly, but fail the first time only when tested by Blin, then when Blin test the module for flappiness it always succeeds

Yeah, this kind of stuff is a bit hard to deal with :)

let me know if any changes needed and when all good I will submit PR

The BUILD logic doesn't spark joy and I would prefer to have two different classes instead. I'll be away till Monday and will be able to merge after that.

[hythm7]

The BUILD logic doesn't spark joy and I would prefer to have two different classes instead.

Fair enough. I have separated BUILD() logic to two classes and submitted PR.

[hythm7]

Thanks for CUR::Staging now Pakku avoids double precompilation.

Fixed in Version ava-1.