Race issue with Exporter module?

[kanavin]

Every now and then the following error happens, when building perl using perl-cross in Yocto project. I am not a specialist, but it seems as though dist/Exporter is racing against other modules in dist?

https://autobuilder.yoctoproject.org/typhoon/#/builders/62/builds/137/steps/7/logs/step1b

The crux of the issue is here:

Exporter.pm did not return a true value at /home/pokybuild/yocto-worker/edgerouter/build/build/tmp/work/mips64-poky-linux/perl/5.28.1-r0/perl-5.28.1/lib/ExtUtils/Command.pm line 5.
| Compilation failed in require.
| BEGIN failed--compilation aborted.
| make[1]: *** [blib/script/.exists] Error 255
| make[1]: Leaving directory `/home/pokybuild/yocto-worker/edgerouter/build/build/tmp/work/mips64-poky-linux/perl/5.28.1-r0/perl-5.28.1/dist/I18N-LangTags'
| make: *** [dist/I18N-LangTags/pm_to_blib] Error 2
| make: *** Waiting for unfinished jobs....
| Exporter.pm did not return a true value at /home/pokybuild/yocto-worker/edgerouter/build/build/tmp/work/mips64-poky-linux/perl/5.28.1-r0/perl-5.28.1/lib/ExtUtils/Command.pm line 5.
| Compilation failed in require.
| BEGIN failed--compilation aborted.
| cp lib/encoding/warnings.pm ../../lib/encoding/warnings.pm
| cp lib/Exporter/Heavy.pm ../../lib/Exporter/Heavy.pm
| cp lib/Exporter.pm ../../lib/Exporter.pm
| make[1]: *** [blib/script/.exists] Error 255
| make[1]: *** Waiting for unfinished jobs....
| make[1]: Leaving directory `/home/pokybuild/yocto-worker/edgerouter/build/build/tmp/work/mips64-poky-linux/perl/5.28.1-r0/perl-5.28.1/dist/Devel-SelfStubber'
| make[1]: Leaving directory `/home/pokybuild/yocto-worker/edgerouter/build/build/tmp/work/mips64-poky-linux/perl/5.28.1-r0/perl-5.28.1/dist/Exporter'

[arsv]

I cannot reproduce it but I think I see what happens there. If possible, try running the build with this patch:

https://github.com/arsv/perl-cross/commit/3e2c1ddd06be97ba75104b1be4b6fdbd08e16bbe

[kanavin]

Thanks for the quick reply! I will let you know when the yocto build machine has given its verdict. It's difficult to establish whether the issue is truly fixed though, as it happens only occasionally.

[kanavin]

Sadly, there seems to be another race:

| gcc  -o miniperl miniperlmain.host.o av.host.o scope.host.o doop.host.o doio.host.o dump.host.o gv.host.o hv.host.o mg.host.o reentr.host.o mro_core.host.o perly.host.o pp.host.o pp_hot.host.o pp_ctl.host.o pp_sys.host.o regcomp.host.o regexec.host.o utf8.host.o sv.host.o taint.host.o toke.host.o util.host.o deb.host.o run.host.o universal.host.o pad.host.o globals.host.o keywords.host.o perlio.host.o perlapi.host.o numeric.host.o mathoms.host.o locale.host.o pp_pack.host.o pp_sort.host.o caretx.host.o dquote.host.o time64.host.o opmini.host.o perlmini.host.o -lm -lcrypt -ldl
| ln -sf libperl.so.5.28.0 libperl.so.5
| ln -sf perldelta.pod pod/perl5281delta.pod
| ./miniperl_top make_patchnum.pl
| ./miniperl_top make_ext_Makefile.pl ext/ExtUtils-Miniperl/Makefile.PL
| Prototype mismatch: sub main::BEGIN () vs none at make_patchnum.pl line 3.
| Constant subroutine BEGIN redefined at make_patchnum.pl line 3.
| Missing or undefined argument to require at make_patchnum.pl line 3.
| BEGIN failed--compilation aborted at make_patchnum.pl line 3.
| Attempt to free unreferenced scalar: SV 0x1a58428 at make_patchnum.pl line 3.
| Makefile:198: recipe for target 'lib/Config_git.pl' failed
| make: *** [lib/Config_git.pl] Error 255
| make: *** Waiting for unfinished jobs....

https://autobuilder.yoctoproject.org/typhoon/#/builders/48/builds/141/steps/7/logs/step1b

[arsv]

Quick test: can you try forcing crosspatch target early? Replacing the build command, make -j 16, with make crosspatch; make -j 16.

Alternatively, try building with this patch: https://github.com/arsv/perl-cross/commit/e1e159a189247af1557fe400ca861714e5ed5af4

The last error makes me think something is really wrong with miniperl, and higher up in the log it starts building objects before it's done patching files.

[kanavin]

Thanks again, will let you know how it goes.

[kanavin]

This time there were no build issues - thanks for looking into it. We'll carry these two patches in the Yocto repo, until they're merged in perl-cross upstream and a new release is made.

[kanavin]

Unfortunately more build races showed up, this time with Dynaloader module! https://autobuilder.yoctoproject.org/typhoon/#/builders/62/builds/170/steps/7/logs/step1b https://autobuilder.yoctoproject.org/typhoon/#/builders/47/builds/168/steps/7/logs/step1b

[kanavin]

I guess dynaloader needs to be also built ahead of other items.

[kanavin]

I've added this patch to our builds, I guess you'll need to pick it up as well. http://git.yoctoproject.org/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/perl-sanity/files/0001-Also-build-dynaloader-separately-as-race-failures-ha.patch?h=akanavin/package-version-updates&id=33cc196d6daf79fe14eeca0db8f737d0cd287736

[arsv]

Yeah, pulled the patch.

[kanavin]

One more race uncovered, this time during 'make install': https://autobuilder.yoctoproject.org/typhoon/#/builders/57/builds/188/steps/7/logs/step1b

Patch is here: http://git.yoctoproject.org/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/perl-sanity/files/0001-Make-sure-install.perl-runs-before-install.man.patch?h=akanavin/package-version-updates

[arsv]

Looking into this. Just making install.man invoke install.perl is probably not a good idea, at this point it should be just install. These targets come from the original perl Makefile and they were supposed to be independent.

[kanavin]

I think install.perl creates the necessary target directory, but install.man does not.

[arsv]

A file, not a directory. $prefix/lib/perl5/5.29.3/x86_64-linux/.packlist. Created by installperl and then read by installman.

https://github.com/arsv/perl-cross/commit/da8c2ac367183138fc1bb15931888471c4f0a3f1

Care should be taken here, these targets are not completely internal and some people use them directly. make install.perl install.sym instead of make install. I'm not sure whether anyone needs install.man but I guess it would be better to keep compatibility for now.

[arsv]

Changes merged, perl-cross-1.2.2 tagged.

[kanavin]

Thanks! I have queued a patch to update to that version: http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=akanavin/package-version-updates&id=d77366c001f80969bd02d63df18ab59134976952