miyagawa / Carmel

CPAN Artifact Repository Manager
Other
138 stars 17 forks source link

Modules not taken from base perl module directory #98

Closed bbrtj closed 1 year ago

bbrtj commented 1 year ago

Can't find any documentation on it, so I'll assume it's a bug:

$ perl -e 'use CryptX; print "ok\n"'
ok
$ carmel exec perl -e 'use CryptX; print "ok\n"'
Can't locate CryptX.pm in @INC (you may need to install the CryptX module) (@INC contains: Carmel::Runtime::FastINC=HASH(0x800bf3df8) /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Beam-Wire-1.025/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Beam-Emitter-1.007/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Module-Runtime-0.016/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Module-Build-0.4231/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Moo-2.005004/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Class-Method-Modifiers-2.13/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Role-Tiny-2.002004/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Sub-Quote-2.006006/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Type-Tiny-2.000001/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Exporter-Tiny-1.004004/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Beam-Service-0.001/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Config-Any-0.32/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Module-Pluggable-5.2/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Data-DPath-0.58/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Class-XSAccessor-1.19/blib/arch /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Class-XSAccessor-1.19/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Iterator-Util-0.02/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Exception-Class-1.45/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Class-Data-Inheritable-0.09/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Devel-StackTrace-2.04/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Iterator-0.03/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Sub-Exporter-0.988/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Data-OptList-0.112/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Params-Util-1.102/blib/arch /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Params-Util-1.102/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Sub-Install-0.928/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/aliased-0.34/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Module-Build-Tiny-0.039/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/ExtUtils-Config-0.008/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/ExtUtils-Helpers-0.026/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/ExtUtils-InstallPaths-0.012/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Path-Tiny-0.124/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Throwable-1.000/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/YAML-1.30/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Import-Into-1.002005/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Mojolicious-9.28/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/MooX-Types-MooseLike-0.29/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Mooish-AttributeBuilder-1.000/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Moose-2.2201/blib/arch /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Moose-2.2201/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Class-Load-0.25/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Module-Implementation-0.09/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Try-Tiny-0.31/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Package-Stash-0.40/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Dist-CheckConflicts-0.11/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Package-Stash-XS-0.30/blib/arch /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Package-Stash-XS-0.30/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Class-Load-XS-0.10/blib/arch /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Class-Load-XS-0.10/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Devel-GlobalDestruction-0.14/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Sub-Exporter-Progressive-0.001013/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Devel-OverloadInfo-0.007/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/MRO-Compat-0.15/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Eval-Closure-0.14/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Module-Runtime-Conflicts-0.003/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Package-DeprecationManager-0.17/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Sub-Name-0.26/blib/arch /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Sub-Name-0.26/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Mouse-v2.5.10/blib/arch /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Mouse-v2.5.10/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Module-Build-XSUtil-0.19/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Devel-CheckCompiler-0.07/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Sub-HandlesVia-0.044/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Role-Hooks-0.008/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Type-Tiny-XS-0.025/blib/arch /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Type-Tiny-XS-0.025/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/namespace-autoclean-0.29/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/B-Hooks-EndOfScope-0.26/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Variable-Magic-0.63/blib/arch /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Variable-Magic-0.63/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Sub-Identify-0.14/blib/arch /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/Sub-Identify-0.14/blib/lib /home/devenv/.carmel/5.37.5-amd64-freebsd/builds/namespace-clean-0.27/blib/lib Carmel::Runtime::SiteINC=HASH(0x800b3a8b8) Carmel::Runtime::SiteINC=HASH(0x800b3de88) /root/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0/amd64-freebsd /root/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0) at -e line 1.
BEGIN failed--compilation aborted at -e line 1.
$

I can reproduce with for example MooseX::XSConstructor, MooX::XSAccessor, but not with Path::Tiny:

$ perl -e 'use Path::Tiny; print "ok\n"'
ok
$ carmel exec perl -e 'use Path::Tiny; print "ok\n"'
ok
$

Same with use Carmel::Setup inside the script. Modules are loadable before that statement, but not after.

miyagawa commented 1 year ago

I can reproduce with for example MooseX::XSConstructor, MooX::XSAccessor

These two modules do not exist on CPAN, perhaps typos?

I did the guess work and tried it with MooseX::XSAccessor, and it seems to work fine:

➜  cat cpanfile
requires 'MooseX::XSAccessor';
➜  carmel install
Using MooseX::XSAccessor (0.009)
Using Class::XSAccessor (1.19)
Using Moose (2.2201)
Using Class::Load (0.25)
Using Data::OptList (0.112)
Using List::Util (1.63)
Using Params::Util (1.102)
Using Sub::Install (0.928)
Using Module::Implementation (0.09)
Using Module::Runtime (0.016)
Using Module::Build (0.4231)
Using Try::Tiny (0.31)
Using Package::Stash (0.40)
Using Dist::CheckConflicts (0.11)
Using Package::Stash::XS (0.30)
Using Class::Load::XS (0.10)
Using Devel::GlobalDestruction (0.14)
Using Sub::Exporter::Progressive (0.001013)
Using Devel::OverloadInfo (0.007)
Using MRO::Compat (0.15)
Using Devel::StackTrace (2.04)
Using Eval::Closure (0.14)
Using Module::Runtime::Conflicts (0.003)
Using Package::DeprecationManager (0.17)
Using Sub::Name (0.26)
Using Sub::Exporter (0.988)
---> Complete! 1 cpanfile dependencies. 26 modules installed.
➜  carmel exec perl -e 'use MooseX::XSAccessor (); warn MooseX::XSAccessor->VERSION'
0.009 at -e line 1.
miyagawa commented 1 year ago

I wonder if you have some environment variables that's affecting where to load the arch-dependent modules from.

bbrtj commented 1 year ago

Sorry, I got the names mixed up. It's MooX::XSConstructor and MooseX::XSAccessor

bbrtj commented 1 year ago

my env is rather simple:

$ printenv
MAIL=/var/mail/devenv
PATH=/root/perl5/perlbrew/perls/perl-5.36.0/bin:/root/perl5/perlbrew/bin:/root/.rakubrew/bin:/root/.rakubrew/versions/moar-2022.04/install/bin:/root/.rakubrew/repos/zef/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/home/devenv/bin
OLDPWD=/usr/home/devenv
PWD=/home/devenv/prj/tests
TERM=screen-256color
HOME=/home/devenv
USER=devenv
SHELL=/bin/sh
BLOCKSIZE=K
$
miyagawa commented 1 year ago

looking at your error output here, the list of paths doesn't include CryptX artifacts.

Did you add the line requires "CryptX"; to your cpanfile, then run carmel install? Every time you add or change a dependency, you have to modify these lines in cpanfile and run carmel install; that's how Carmel maintains the state of your dependencies in the .snapshot file.

Please upload the entire cpanfile,cpanfile.snapshot and .carmel/MySetup.pm if that doesn't resolve the issue.

miyagawa commented 1 year ago

My .carmel/MySetup.pm looks like this after adding requires "MooX::XSConstructor"; to cpanfile, and running carmel install:

➜  head -30 .carmel/MySetup.pm
# DO NOT EDIT! Auto-generated via carmel install.
package Carmel::MySetup;

our %environment = (
  'inc' => [
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/MooX-XSConstructor-0.002/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Class-XSConstructor-0.009/blib/arch',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Class-XSConstructor-0.009/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Exporter-Tiny-1.006000/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Scalar-List-Utils-1.63/blib/arch',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Scalar-List-Utils-1.63/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Ref-Util-0.204/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Ref-Util-XS-0.117/blib/arch',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Ref-Util-XS-0.117/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Hook-AfterRuntime-0.006/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/B-Hooks-Parser-0.21/blib/arch',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/B-Hooks-Parser-0.21/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/B-Hooks-OP-Check-0.22/blib/arch',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/B-Hooks-OP-Check-0.22/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/ExtUtils-Depends-0.8001/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Module-Build-0.4231/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Moo-2.005004/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Class-Method-Modifiers-2.13/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Role-Tiny-2.002004/blib/lib',
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Sub-Quote-2.006006/blib/lib'
  ],
  'path' => [
    '/Users/miyagawa/.carmel/5.34.1-darwin-2level/builds/Module-Build-0.4231/blib/script'
  ],
  'execs' => {
bbrtj commented 1 year ago

CryptX is not in my cpanfile. My problem is that CryptX was installed globally in perl using cpanm CryptX, so I assume it should also be loadable under carmel exec without requires "CryptX"; in the cpanfile. If that isn't the case, then is it a documented behavior?

miyagawa commented 1 year ago

My problem is that CryptX was installed globally in perl using cpanm CryptX

Yes, that is indeed the problem. To make sure you're NOT accidentally using locally-installed modules, Carmel (like Carton) clears out the site installation path, so that you can run carmel install and run carmel exec on another machine, to satisfy the prerequisite without relying on locally-updated modules + versions.

You have to add all the non-core modules to cpanfile and run carmel install.

miyagawa commented 1 year ago

If you have a strong reason to avoid adding some specific modules to cpanfile while understanding the risk of doing so (e.g. have to install modules that's not available on CPAN), you can:

This will not clear the site_perl path from INC, so that your locally-installed modules are still available. This is documented in Carmel.pm under the production deployment section.

https://metacpan.org/pod/Carmel#Production-Deployments

bbrtj commented 1 year ago

My first test with Path::Tiny was indeed flawed - it is a popular distribution, and got pulled by Carmel as an indirect dependency. I checked with other non-core pure perl module and now I see this behavior is consistent (it does not load).

It isn't immediately obvious that carmel exec / use Carmel::Setup; does that, but it makes sense. Thank you for the explanation!