Test failures under latest bleadperl

[hvds]

It looks like the initial divergence is occurring because Exporter now has use strict, which means it goes down different pathways. My initial assumption is that this is the only thing that has changed - but you can expect more changes of the same nature to land, as part of a drive to make code shipped with perl as far as possible strict and warnings compliant.

See http://fast-matrix.cpantesters.org/?dist=Devel-NYTProf-6.06;os=linux;perl=5.33.3;reports=1 for examples.

[hvds]

Note that this was reported to p5p under https://github.com/Perl/perl5/issues/18215, please poke us there if you believe perl core has actually done something wrong here.

[timbunce]

Realistically I'm not going to have time to re-climb the learning curve I'd need to address this myself. I'd be happy to assist others with pointers though.

[timbunce]

As a general point, NYTProd tests are too sensitive. You can see in the history various instances of past failures caused by perl changes and the various ways there were addressed.

[jkeenan]

As a general point, NYTProd tests are too sensitive. You can see in the history various instances of past failures caused by perl changes and the various ways there were addressed.

Is there any real bug in NYTProf here -- or is it merely that the expectations set in t/test62-subcaller1.t need to be updated?

Thank you very much. Jim Keenan

[wolfsage]

Looking at the failures, things seem to make sense - it's all about including strict in the calculations, but a couple of failures are of this variety:

--- test62-subcaller1.calls     2021-03-22 15:19:00.657183957 -0400
+++ test62-subcaller1.calls_new 2021-03-22 15:27:18.057845601 -0400
@@ -1,8 +1,12 @@
+main::CORE:substcont 3
+main::sub4 2
+Exporter::BEGIN@3 1
+Exporter::BEGIN@3;strict::import 1
+Exporter::BEGIN@4 1
+Exporter::BEGIN@4;strict::unimport 1
 Devel::NYTProf::Test::example_xsub 2
 main::sub1 1
 main::CORE:sort 2
 main::CORE:sort;Devel::NYTProf::Test::example_xsub 3
 main::CORE:sort;main::sub2 6
 main::CORE:subst 1
-main::CORE:substcont 3
-main::sub4 2
ok 65

And I'm not sure why things are being reordered there. @timbunce Any ideas why the order in this file changes and what that means? I had assumed this was 99% just regenerating the rdt files uses by the tests (helpfully provided as _new) versions but this one thing is confounding me

[jkeenan]

Looking at the failures, things seem to make sense - it's all about including strict in the calculations, but a couple of failures are of this variety:

--- test62-subcaller1.calls     2021-03-22 15:19:00.657183957 -0400
+++ test62-subcaller1.calls_new 2021-03-22 15:27:18.057845601 -0400
@@ -1,8 +1,12 @@
+main::CORE:substcont 3
+main::sub4 2
+Exporter::BEGIN@3 1
+Exporter::BEGIN@3;strict::import 1
+Exporter::BEGIN@4 1
+Exporter::BEGIN@4;strict::unimport 1
 Devel::NYTProf::Test::example_xsub 2
 main::sub1 1
 main::CORE:sort 2
 main::CORE:sort;Devel::NYTProf::Test::example_xsub 3
 main::CORE:sort;main::sub2 6
 main::CORE:subst 1
-main::CORE:substcont 3
-main::sub4 2
ok 65

And I'm not sure why things are being reordered there. @timbunce Any ideas why the order in this file changes and what that means? I had assumed this was 99% just regenerating the rdt files uses by the tests (helpfully provided as _new) versions but this one thing is confounding me

I've made some progress in reducing the number of failing unit tests within t/test62-subcaller1-b.rdt.

https://github.com/jkeenan/devel-nytprof/tree/test62-failures-20210321

In the branch above, I have split t/test62-subcaller1.rdt into '-a' and '-b' variants. '-a' is for pre-breakage perls; '-b' is for post-breakage perls. This is very much a hack in progress.

[timbunce]

Thanks for working on this @jkeenan!

[jkeenan]

Thanks for working on this @jkeenan!

You're welcome, Tim. Please take a look at https://github.com/timbunce/devel-nytprof/pull/147. That's as far as I've gotten, as I don't know how to deal with those machine-specific absolute paths. Cf. https://travis-ci.org/github/jkeenan/devel-nytprof/builds.

[jkeenan]

Pull request https://github.com/timbunce/devel-nytprof/pull/147 has now advanced to the point where it should resolve the problem discussed in this issue.

Thank you very much. Jim Keenan

[jkeenan]

This p.r. was developed on FreeBSD-12; hence, tests pass there. Also successfully tested on OpenBSD-6.6.