search

LMS-Community / slimserver-vendor

Third-party software used with Lyrion Music Server
https://lyrion.org
42 stars 68 forks source link

build script - make sure so and pm versions match #35

Closed dnalor closed 6 years ago

dnalor commented 7 years ago

When the build script buildme.sh builds modules with a newer version than the ones included in the Logitech/slimserver/tree/public/7.9/CPAN directory, the pm file(s) matching the binary so file also need to be copied to the build directory.

This is done for Class::XSAccessor Compress::Raw::Zlib DBI JSON::XS

but not for Image::Scale (except for FreeBSD / perl > 5.22) (see #8 ) YAML::LibYAML (for perl > 5.26)

So either

fsbruva commented 7 years ago

While this approach will work for an individual, it may not necessarily work for all. This is because unless a module was updated from its original version, the legacy .pm already exists in CPAN, so we'd be duplicating it in CPAN/arch. But you bring a valid point that I had forgotten needed fixing. @mherger, I can develop a PR to address this.

dnalor commented 7 years ago

While this approach will work for an individual, it may not necessarily work for all. This is because unless a module was updated from its original version, the legacy .pm already exists in CPAN, so we'd be duplicating it in CPAN/arch.

If you would have checked either the tree or the binary deb/rpm/tgz packages, you would have noticed that they already include the .pm files of both versions for all the modules I mentioned above - the perl module loader is smart enough to use the correct one. The only thing missing is for the build script to automatically copy the new .pm to the build/arch/ directory, in case it is copied into a LMS install from a logitechmediaserver-x.y.z-noCPAN.tgz tarball, which doesn't include the CPAN/arch directory

fsbruva commented 7 years ago

If you would have checked either the tree

Same team here. No need for this.

Your fix would replicate EVERY .pm into arch, right? Not just the updated ones?

Additionally, I was operating under the assumption that the directory structure produced by the build script ought to conform to that promulgated in server. In #14, @mherger indicated a desire to place the updated files down lower In the directory tree. He simultaneously gave your suggestion a thumbs up. Not sure which direction he wants. The answer to this will inform/direct future module updates (I saw you have many in your custom build script).

mherger commented 6 years ago

@fsbruva - tbh I don't plan to re-build anything. These builds I try to do once and never look back... Which obviously can lead to this kind of duplicated code. Re-building and making sure all those new binaries actually do work is an effort I can't provide. So... there's kind of a base-line, which covers the oldest builds. Some later perl versions might require a new module version. If it's the same for all platforms of that perl version, I put the perl files in the perl version's folder. If there's an even more recent module version being built for one particular perl version & platform combination, I put it in that platform's sub-folder.

Yes, this does result in duplicates. But oh well... compared to the binaries we carry around those perl files are peanuts.

fsbruva commented 6 years ago

Peanuts are yummy.

Thanks for the insight. I'll just make a command line knob to control the build behavior. Easy peasy.

fsbruva commented 6 years ago

@dnalor , @mherger , I am still working this, and need some feedback/guidance: Would either of you consider it unacceptable behavior if the only .pm files that were present in the destination (build/arch, build/arch/5.x, or build/arch/5.x/{arch-OS}) were from the latest run of the build script? I think I know what your answer(s) will be, but I appreciate your insight.

mherger commented 6 years ago

I'm not sure I understand your question.

fsbruva commented 6 years ago

Sorry - I had been attempting to selectively replicate the necessary .pm files for only those modules that were using updated versions from what's in server/CPAN (DBI, Image::Scale, etc), and place them all someplace (like you do over in server). The challenge I was running into was the rsync for the binaries during a subsequent build run was blowing away the .pms of the previous run. That is, with my changes, you build everything, and all the .pms would be there. If you ran the build again* for a single module (DBI), only the DBI .pms would be replicated. This exactly conforms to the current build script behavior - the .pms for a particular module are copied into arch if that module was build during the run, but all existing .so files get copied into arch every time. Three questions to frame my work:

  1. Do you prefer that all .pm files be copied every run (to match the .so copy behavior)?
  2. Do you want the build script to (optionally) produce the directory structure you have in server?
  3. Do you prefer that only the .pm files for new modules are copied? Or is the potential replication of the Perl version independent modules files in server/CPAN acceptable?
mherger commented 6 years ago

TBH I don't really care how DIYers would store those files. For the "official" builds I try to keep the versions across one Perl version identical, just to prevent this kind of mess. I therefore put the pure Perl parts in the upper level. Eg. I don't mix Image::Scale versions within the same Perl version. I'd start to use newer versions once I start to build for a new Perl. Or I re-compile it for all platforms of a given Perl version.

Most people using this script are DIYers, though. For them it might be the easiest to just include the pure Perl parts with the binaries.

fsbruva commented 6 years ago

Sorry for bombing everyone's email with my troglodytic GIT tactics. I had to do numerous rebasing operations, and each iteration had the issues ID'd in the commit messages.

dnalor commented 6 years ago

@fsbruva

The challenge I was running into was the rsync for the binaries during a subsequent build run was blowing away the *.pms of the previous run. That is, with my changes, you build everything, and all the .pms would be there. If you ran the build again for a single module (DBI), only the DBI .pms would be replicated. This exactly conforms to the current build script behavior - the .pms for a particular module are copied into arch if that module was build during the run, but all existing .so files get copied into arch every time.

I'm still not sure I understand the issue. First you seem to suggest that the all pms from a previous build get deleted when building a single module, then you write that this conforms to the current build script behaviour (which is not to delete them). My expectation would be that building a module (either on it's own, or as part of building all) results in all files (pms, sos) required by the runtime being copied in the correct structure to the build/arch directory, ready to be installed into LMS's CPAN/arch directory. This should have no side-effects (e.g. affect other modules build results, like deleting any pms already installed in build/arch), and is the current script behaviour (except for Image::Scale and YAML::LibYAML)

fsbruva commented 6 years ago

Sorry for the confusion. Based on my testing, I saw the contents of PERL_ARCH/ got blown away every time. The rest of the stuff in perl5/lib stays between build. So, all the binaries got copied every run, but even the results of the previous cp -rP (blah) into PERL_ARCH/ would get overwritten by the next run of the build script. Maybe I'm mistaken. It really doesn't matter the way it was, because the PR I submitted conforms to your (and Michael's) desired/expected behavior. Everything gets copied everytime.

fsbruva commented 6 years ago

@dnalor Can you please give aa71ba7 a try, and let me know if this issue can be closed?

dnalor commented 6 years ago

@fsbruva Sorry for the delay. Looks fine to me (the --include=*.so in the second rsync is superfluous, but probably won't hurt). Closing the issue - thanks.