Don't bundle CPAN libraries

[phile314]

I am trying to package slimserver for NixOS. However, I noticed that the repository contains a great amount of copied CPAN libraries. This makes it difficult to package slimserver, as some CPAN packages need patches to work properly on NixOS. Furthermore, using copies also adds a significant security risk as slimserver may be using outdated/vulnerable library versions.

It would be great if slimserver instead could use CPAN packages directly and not use the bundled versions. I am aware that this probably is a lot of work, but it would be a great help.

I guess this could also help the Arch package @stefansielaff ?

[mherger]

Most of the CPAN modules might not be a problem at all: just dump the CPAN sub-folder. You might need to wipe the modules.conf, too.

The other issue is the lib folder: the modules in there have been modified to fix issues or adjust to some need.

Now that said: it might just work with standard modules. But we can't guarantee. We're aware that the situation isn't great, but it's been working rather fine for years...

[stefansielaff]

As a developer (perl is not my predilection) I understand both sides and deal with it. This is why I've forked this a while ago and use the fork for the arch packages instead of the original repo. I'm trying to remove as many modules as possible over time, but to be honest this is a brave approach because Arch moves quickly and you can never rely on anything.

[phile314]

Just to let you know, just ignoring most of the CPAN modules did work fine. The only two packages that I used from the bundled copies where DBIx and SQL, as it seems that slimserver doesn't work with the current versions of those packages.

If anyone is interested, this are my build specifications: Package: https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/slimserver/default.nix Service: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/audio/slimserver.nix#L45

[mherger]

@phile314 - that's good to know. Thanks!

[stefansielaff]

@phile314 I'm late but just read your reply... interesting! There are a few more modules in the delivered folder which don't seem to be defined in your script. Starting with Algorithm::C3 - have you simply dropped them or are they pulled in as a dependency on NixOS?

[phile314]

If they are transitive dependencies of some other dependency the get pulled in, else not. I just kept adding libraries until things started working. It could be that I am missing some libraries and some functionality might be broken. Haven't noticed any breakage yet though.

[paulgevers]

I concur that it seems to be working with SQL and DBIx from slimserver with the rest removed. Without those I get errors like below. I haven't checked yet if it is either SQL or DBIx or both. Either way, it seems modules.conf should be updated with a maximum version to reflect this. The upper limit for Audio::Scan seems unneeded though as 0.98 seems to work.

[17-11-30 22:14:20.7100] main::init (387) Starting Logitech Media Server (v7.9.1, 1511690631, Sun Nov 26 10:03:51 UTC 2017) perl 5.026001 - x86_64-linux-gnu-thread-multi
[17-11-30 22:14:22.9816] Slim::Utils::Misc::msg (1250) Warning: [22:14:22.9809] DBIx::Class::Componentised::inject_base(): Use of DBIx::Class::UTF8Columns is strongly discouraged. See documentation of DBIx::Class::UTF8Columns for more info
[17-11-30 22:14:23.7313] Slim::Schema::Storage::throw_exception (122) Error: DBI Exception: DBD::SQLite::db do failed: database persistentdb is already in use
[17-11-30 22:14:23.7330] Slim::Schema::Storage::throw_exception (122) Backtrace:

   frame 0: Slim::Utils::Log::logBacktrace (/usr/share/perl5/Slim/Schema/Storage.pm line 122)
   frame 1: Slim::Schema::Storage::throw_exception (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1501)
   frame 2: DBIx::Class::Storage::DBI::__ANON__ (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1468)
   frame 3: DBIx::Class::Storage::DBI::__ANON__ (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 856)
   frame 4: DBIx::Class::Storage::DBI::__ANON__ (/usr/share/perl5/DBIx/Class/Storage/BlockRunner.pm line 130)
   frame 5: DBIx::Class::Storage::BlockRunner::try {...}  (/usr/share/perl5/Try/Tiny.pm line 103)
   frame 6: (eval) (/usr/share/perl5/Try/Tiny.pm line 94)
   frame 7: Try::Tiny::try (/usr/share/perl5/DBIx/Class/Storage/BlockRunner.pm line 134)
   frame 8: DBIx::Class::Storage::BlockRunner::__ANON__ (/usr/share/perl5/Context/Preserve.pm line 23)
   frame 9: Context::Preserve::preserve_context (/usr/share/perl5/DBIx/Class/Storage/BlockRunner.pm line 213)
   frame 10: DBIx::Class::Storage::BlockRunner::_run (/usr/share/perl5/DBIx/Class/Storage/BlockRunner.pm line 105)
   frame 11: DBIx::Class::Storage::BlockRunner::run (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 857)
   frame 12: DBIx::Class::Storage::DBI::dbh_do (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1470)
   frame 13: DBIx::Class::Storage::DBI::_do_query (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1426)
   frame 14: DBIx::Class::Storage::DBI::connect_call_do_sql (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1391)
   frame 15: DBIx::Class::Storage::DBI::try {...}  (/usr/share/perl5/Try/Tiny.pm line 103)
   frame 16: (eval) (/usr/share/perl5/Try/Tiny.pm line 94)
   frame 17: Try::Tiny::try (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1419)
   frame 18: DBIx::Class::Storage::DBI::_do_connection_actions (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1401)
   frame 19: DBIx::Class::Storage::DBI::try {...}  (/usr/share/perl5/Try/Tiny.pm line 103)
   frame 20: (eval) (/usr/share/perl5/Try/Tiny.pm line 94)
   frame 21: Try::Tiny::try (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1419)
   frame 22: DBIx::Class::Storage::DBI::_do_connection_actions (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1065)
   frame 23: DBIx::Class::Storage::DBI::_run_connection_actions (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 1058)
   frame 24: DBIx::Class::Storage::DBI::_populate_dbh (/usr/share/perl5/DBIx/Class/Storage/DBI.pm line 947)
   frame 25: DBIx::Class::Storage::DBI::ensure_connected (/usr/share/perl5/Slim/Schema/Storage.pm line 43)
   frame 26: (eval) (/usr/share/perl5/Slim/Schema/Storage.pm line 43)
   frame 27: Slim::Schema::Storage::dbh (/usr/share/perl5/Slim/Schema.pm line 287)
   frame 28: Slim::Schema::_connect (/usr/share/perl5/Slim/Schema.pm line 130)
   frame 29: Slim::Schema::init (/usr/sbin/squeezeboxserver line 552)
   frame 30: main::init (/usr/sbin/squeezeboxserver line 675)
   frame 31: main::main (/usr/sbin/squeezeboxserver line 1213)

[fsbruva]

I concur that it seems to be working with SQL and DBIx from slimserver with the rest removed. Without those I get errors like below. I haven't checked yet if it is either SQL or DBIx or both.

This is a known issue with DBIx, as documented in this thread.

@mherger Is the scope to make slimserver detect & use newer DBIx known? Is it simply not worth the effort?

[mherger]

I never looked into this. Feel free to give it a try. But if Andy at the time didn't want to move ahead, then I think it might be non-trivial.

[adamcstephens]

From my testing, the vast majority of the vendored modules can be replaced with CPAN modules. In nixpkgs we recently bumped perl to 5.38 which broke slimserver, but I've since proven that it can work with 5.38. We'd rather not use the slimserver-vendor repo or built in compiled modules if possible. https://github.com/NixOS/nixpkgs/pull/265556

[gordonb3]

Just stumbled across this. I am maintaining a package for Gentoo Linux as part of an ARMv5 based home server operating system. Like you I found that LMS is seriously picky about the DBIx version. I have not detected any issue with SQL. Initially I ran the slimserver-vendor routine to build the arch specific modules but I didn't like it one bit that some of the modules required literally hours to compile as they statically linked to antique versions of e.g. ffmpeg.

I found that most of the required modules for running LMS were also present as system provided packages (referencing CPAN) and in some cases even already installed as part of the OS web based GUI. Which was yet another nuisance as Perl prefers system path for loading libraries but LMS sets the module load path to prefer its own CPAN and Slim folders - which leads to cryptic error messages about versions being incorrect.

I created additional packages for the missing modules from slimserver-vendor, using the system default of dynamic linking to modern support libraries. The result is that my LMS installation does not contain any arch specific modules at all, however this does not eliminate all of the conflicts I mentioned above. To accomplish that my final step in installing LMS is a cyclic removal of every module instance within the CPAN folder that also exists in the Perl system path (except of course DBIx). I have stepped up from 5.34 to 5.36 and 5.38 without needing to make any changes to this routine.

[adamcstephens]

The list we're keeping from the modules vendored in this repo are listed here: https://github.com/NixOS/nixpkgs/blob/776dfd7eaab2aa41d7cb1c6c0619e04032fe0535/pkgs/servers/slimserver/default.nix#L114-L125

I did end up using the AudioScan module version from slimserver-vendor, as its newer than CPAN and some users were experiencing problems with the CPAN version. https://github.com/NixOS/nixpkgs/blob/f9d39fb9aff0efee4a3d5f4a6d7c17701d38a1d8/pkgs/top-level/perl-packages.nix#L1295-L1310

[gordonb3]

Yes. CPAN is stuck at 1.01 As I didn't want to break the Gentoo package method for Perl modules and thus risk Audio::Scan to stay behind on a Perl upgrade I added a patch to version 1.02 in my repository. I've been adding quite a few more patches the last year...

https://github.com/gordonb3/bubba-overlay/tree/master/dev-perl/Audio-Scan