zef installs some module resources to /tmp/.zef which are missing on reboot

avuserow commented 7 months ago

zef install Crypt::Argon2 installs the resources (such as its .so file) to /tmp. Since /tmp is tmpfs on many Linux distros (including Fedora), this is removed upon a reboot, and results in errors such as

Cannot locate native library '/tmp/.zef/1703642021.119893/1703642022.119893.237.1946489885046/resources/34941271E1DC62CAA3FE862F03D60D58AB930360.so': /tmp/.zef/1703642021.119893/1703642022.119893.237.1946489885046/resources/34941271E1DC62CAA3FE862F03D60D58AB930360.so: cannot open shared object file: No such file or directory

Context

I have not tested this with many modules. I know that Crypt::Argon2 is affected and my own Audio::TagLib is not, despite both creating .so files. Argon2 uses LibraryMake while TagLib does not so that may be a potential difference. The variables in Build.pm for Crypt::Argon2 look fine so it may be something within LibraryMake.

I was able to determine that d88d0ea5f118780f6ffcdee6690d7776e2b3aaa6 is the first bad commit with git bisect.

Expected Behavior

Module resources should be installed to something like <SOME_PREFIX>/install/share/perl6/site/resources

Actual Behavior

Module dependencies are installed in /tmp/.zef

Steps to Reproduce

zef install Crypt::Argon2
rm -rf /tmp/.zef # simulate clearing tmpfs
# try to use it:
raku -e'use Crypt::Argon2::DeriveKey; say argon2-derive-key("something", :hashlen(16))'

Your Environment

raku -v

Welcome to Rakudo™ v2023.11.
Implementing the Raku® Programming Language v6.d.
Built on MoarVM version 2023.11.

zef list --installed

App::Mi6:ver<3.0.4>:auth<zef:skaji>
Audio::TagLib:ver<0.0.6>:auth<zef:avuserow>
Crypt::Argon2:ver<0.2.1>:auth<github:skinkade>
Crypt::Random:ver<0.4.1>:auth<github:skinkade>
Crypt::TweetNacl:ver<0.0.4>
DBIish:ver<0.6.6>:auth<zef:raku-community-modules>:api<1>
Distribution::Builder::MakeFromJSON:ver<0.6>:auth<github:niner>
File::Directory::Tree:auth<labster>
File::Find:ver<0.2.1>:auth<zef:raku-community-modules>
File::Temp:ver<0.0.10>:auth<zef:rbt>
File::Which:ver<1.0.4>
JSON::Class:ver<0.0.21>:auth<zef:jonathanstowe>:api<1.0>
JSON::Fast:ver<0.19>:auth<cpan:TIMOTIMO>
JSON::Marshal:ver<0.0.25>:auth<zef:jonathanstowe>:api<1.0>
JSON::Name:ver<0.0.7>:auth<zef:jonathanstowe>:api<1.0>
JSON::OptIn:ver<0.0.2>:auth<zef:jonathanstowe>
JSON::Tiny:ver<1.0>
JSON::Unmarshal:ver<0.15>:auth<zef:raku-community-modules>
LibraryCheck:ver<0.0.12>:auth<zef:jonathanstowe>:api<1.0>
LibraryMake:ver<1.0.4>:auth<zef:jjmerelo>
License::SPDX:ver<3.22.0>:auth<zef:jonathanstowe>:api<1.0>
META6:ver<0.0.30>:auth<zef:jonathanstowe>:api<1.0>
NativeCall::TypeDiag:auth<Sylvain Colinet>
NativeHelpers::Blob:ver<0.1.12>:auth<github:salortiz>
NativeLibs:ver<0.0.9>:auth<github:salortiz>
OO::Monitors:ver<1.1.1>
Path::Finder:ver<0.4.7>:auth<zef:leont>
Pod::Load:ver<0.7.2>:auth<zef:jjmerelo>
Pod::To::HTML:ver<0.8.1>:auth<github:Raku>
Pod::To::Markdown:ver<0.2.1>:auth<github:softmoth>
Readline:ver<0.1.6>:auth<cpan:fooist>
Saiph:ver<0.0.1>:auth<Adrian Kreher>
Shell::Command:ver<1.1>:auth<zef:raku-community-modules>
System::Query:ver<0.1.6>:auth<zef:tony-o>
TAP:ver<0.3.14>
Template::Mustache:ver<1.2.3>:auth<github:softmoth>
Term::termios:ver<0.2.7>:auth<zef:krunen>
Terminal::Getpass:ver<0.0.11>:auth<zef:titsuki>
Test::META:ver<0.0.20>:auth<zef:jonathanstowe>:api<1.0>
Test::Output:ver<1.001006>:auth<zef:raku-community-modules>
URI:ver<0.3.7>:auth<zef:raku-community-modules>
UUID:ver<1.0.0>:auth<github:retupmoca>
fez:ver<55>:auth<zef:tony-o>:api<0>
if:ver<0.1.2>:auth<zef:raku-community-modules>
zef:ver<0.19.1>:auth<github:ugexe>:api<0>
rakudo:ver<2023.11>:auth<Yet Another Society>

Even though zef 19.1 is installed, I noticed this with the latest zef. If you need any help, feel free to ping me on discord if I don't respond quickly here.

ugexe commented 7 months ago

Are you sure those files aren't actually installed in the expected location, and that your code isn't using a precompiled string pointing to /tmp/whatever (the temporary location modules are installed to before actually being moved to the expected install location)?

ugexe commented 7 months ago

Yeah, I think https://github.com/skinkade/p6-crypt-argon2/blob/70b041bdf7ac6d68e7c010d3581e075f9a704d4c/lib/Crypt/Argon2/Base.pm6#L9C1-L9C53 is incorrect (it should not be calling .Str on %?RESOURCES) and should be just constant ARGON2 = %?RESOURCES<libraries/argon2>

ugexe commented 7 months ago

I wonder if it would be worth opening an issue or a PR on rakudo to deprecate calling .Str on a Distribution::Resources object... @niner thoughts?

niner commented 7 months ago

Yeah, this is a recurring issue. Don't know why people insist on stringifying those paths.

avuserow commented 7 months ago

Thanks for the help debugging. I filed a PR for Crypt::Argon2 and verified that it works now.

ugexe / zef