Failed test 'Japanese area name is decoded to Unicode characters too'

[eserte]

On some of my smokers (various freebsd, centos7, fedora systems; various perl versions between 5.8.9 and 5.30.0) the test suite fails like this:

#   Failed test 'Japanese area name is decoded to Unicode characters too'
#   at t/unicode.t line 21.
warning made fatal: Wide character in print at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test2/Formatter/TAP.pm line 125.

 at t/inc/nptestutils.pm line 17.
    nptestutils::__ANON__("Wide character in print at /usr/perl5.22.3p/lib/site_perl/5.2"...) called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test2/Formatter/TAP.pm line 125
    Test2::Formatter::TAP::write(Test::Builder::Formatter=HASH(0x802ed34c8), Test2::Event::Diag=HASH(0x80336c6c0), 2, HASH(0x80336c660)) called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test2/Hub.pm line 373
    Test2::Hub::process(Test2::Hub=HASH(0x802ecc8d0), Test2::Event::Diag=HASH(0x80336c6c0)) called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test2/Hub.pm line 306
    Test2::Hub::send(Test2::Hub=HASH(0x802ecc8d0), Test2::Event::Diag=HASH(0x80336c6c0)) called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test2/API/Context.pm line 272
    Test2::API::Context::send_event(Test2::API::Context=HASH(0x80336c798), "Diag", "message", "         got: undef\x{a}    expected: '\x{5ef6}\x{5ca1}'\x{a}") called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test2/API/Context.pm line 440
    Test2::API::Context::diag(Test2::API::Context=HASH(0x80336c798), "         got: undef\x{a}    expected: '\x{5ef6}\x{5ca1}'\x{a}") called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test/Builder.pm line 1288
    Test::Builder::diag(Test::Builder=HASH(0x801db8330), "         got: undef\x{a}    expected: '\x{5ef6}\x{5ca1}'\x{a}") called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test/Builder.pm line 870
    Test::Builder::_is_diag(Test::Builder=HASH(0x801db8330), undef, "eq", "\x{5ef6}\x{5ca1}") called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test/Builder.pm line 815
    Test::Builder::is_eq(Test::Builder=HASH(0x801db8330), undef, "\x{5ef6}\x{5ca1}", "Japanese area name is decoded to Unicode characters too") called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test/More.pm line 405
    Test::More::is(undef, "\x{5ef6}\x{5ca1}", "Japanese area name is decoded to Unicode characters too") called at t/unicode.t line 21
    (in cleanup) warning made fatal: A context appears to have been destroyed without first calling release().
Based on $@ it does not look like an exception was thrown (this is not always
a reliable test)

This is a problem because the global error variables ($!, $@, and $?) will
not be restored. In addition some release callbacks will not work properly from
inside a DESTROY method.

Here are the context creation details, just in case a tool forgot to call
release():
  File: t/unicode.t
  Line: 21
  Tool: Test::More::is

Here is a trace to the code that caused the context to be destroyed, this could
be an exit(), a goto, or simply the end of a scope:
Context destroyed at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test/Builder.pm line 870.
    Test::Builder::_is_diag(Test::Builder=HASH(0x801db8330), undef, "eq", "\x{5ef6}\x{5ca1}") called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test/Builder.pm line 815
    Test::Builder::is_eq(Test::Builder=HASH(0x801db8330), undef, "\x{5ef6}\x{5ca1}", "Japanese area name is decoded to Unicode characters too") called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test/More.pm line 405
    Test::More::is(undef, "\x{5ef6}\x{5ca1}", "Japanese area name is decoded to Unicode characters too") called at t/unicode.t line 21

Cleaning up the CONTEXT stack...

 at t/inc/nptestutils.pm line 17.
    nptestutils::__ANON__("A context appears to have been destroyed without first callin"...) called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test2/API/Context.pm line 77
    Test2::API::Context::DESTROY(Test2::API::Context=HASH(0x80336c798)) called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Carp.pm line 292
    eval {...} called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Carp.pm line 292
    Test::Builder::diag() called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test/Builder.pm line 870
    Test::Builder::_is_diag(Test::Builder=HASH(0x801db8330), undef, "eq", "\x{5ef6}\x{5ca1}") called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test/Builder.pm line 815
    Test::Builder::is_eq(Test::Builder=HASH(0x801db8330), undef, "\x{5ef6}\x{5ca1}", "Japanese area name is decoded to Unicode characters too") called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test/More.pm line 405
    Test::More::is(undef, "\x{5ef6}\x{5ca1}", "Japanese area name is decoded to Unicode characters too") called at t/unicode.t line 21
    (in cleanup) warning made fatal: A context appears to have been destroyed without first calling release().
Based on $@ it does not look like an exception was thrown (this is not always
a reliable test)

This is a problem because the global error variables ($!, $@, and $?) will
not be restored. In addition some release callbacks will not work properly from
inside a DESTROY method.

Here are the context creation details, just in case a tool forgot to call
release():
  File: t/unicode.t
  Line: 21
  Tool: Test::More::is

Here is a trace to the code that caused the context to be destroyed, this could
be an exit(), a goto, or simply the end of a scope:
Context destroyed at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test/More.pm line 405.
    Test::More::is(undef, "\x{5ef6}\x{5ca1}", "Japanese area name is decoded to Unicode characters too") called at t/unicode.t line 21

Cleaning up the CONTEXT stack...

 at t/inc/nptestutils.pm line 17.
    nptestutils::__ANON__("A context appears to have been destroyed without first callin"...) called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test2/API/Context.pm line 77
    Test2::API::Context::DESTROY(Test2::API::Context=HASH(0x80329bcc0)) called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Carp.pm line 292
    eval {...} called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Carp.pm line 292
    Test::Builder::is_eq() called at /usr/perl5.22.3p/lib/site_perl/5.22.3/Test/More.pm line 405
    Test::More::is(undef, "\x{5ef6}\x{5ca1}", "Japanese area name is decoded to Unicode characters too") called at t/unicode.t line 21
# Tests were run but no plan was declared and done_testing() was not seen.
# Looks like your test exited with 255 just after 2.
t/unicode.t ......................................... 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 1/2 subtests 

[DrHyde]

I'm very glad you reported this, as I've seen failure reports from CPAN-testers a couple of times before but not been able to replicate it. Expect questions!

To start with, can you tell me what locale settings you've got on the machines where this happens?

Also what does this say?

perl -MData::Dumper -MI18N::LangTags::Detect -E 'say Dumper(I18N::LangTags::implicate_supers(I18N::LangTags::Detect::detect()))'

[DrHyde]

Thinking about it a bit more ... the key here seems to be Test::More::is(undef, "\x{5ef6}\x{5ca1}", "Japanese area name ... indicating that this line got an undef from here. That can happen if the country-specific class can't be loaded here. That's something perfectly valid if the "country" is something like Inmarsat, and I can replicate the failure if I just delete the module for Japan.

Did you run out of disk space while untarring the dist?

[eserte]

I'll stick to the CentOS 7 smoker for your questions first:

• locale:

  $VAR1 = 'en-us';
  $VAR2 = 'en';

• disk space: no problem here, after untarring there's still >7GB left

While stepping with the debugger I see:

  DB<6> 
Number::Phone::JP::areaname(/opt/perl-5.30.0/lib/site_perl/5.30.0/Number/Phone/JP.pm:158):
158:    sub areaname   { undef }

[eserte]

Confirmed: just install Number::Phone::JP first, then the Number::Phone test will fail.

[DrHyde]

Ahhh, so obvious. Fixed in 66b7030. I'll push a new release out tomorrow assuming the CI doesn't whine.

Thanks for your help!

[DrHyde]

Released as version 3.8004.