Test-More / test-more

Test2, Test::More, Test::Simple and Test::Builder Perl modules for writing tests
Other
140 stars 88 forks source link

t\Legacy\fail-more.t and t\Legacy\More.t appear to fail when running "nmake test" on Windows #1002

Closed twata1 closed 2 months ago

twata1 commented 2 months ago

Hello,

Since version 1.302193, when I run "nmake test" on Windows, t\Legacy\fail-more.t and t\Legacy\More.t appear to fail. Please see example outputs below.

C:\home\mountain5380rc1\Test-Simple-1.302201>nmake test TEST_FILES="t\Legacy\fail-more.t t\Legacy\More.t"

Microsoft (R) Program Maintenance Utility Version 14.33.31630.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        "C:\perl64\5.38.0rc1\bin\perl.exe" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches;
harness(0, 'blib\lib', 'blib\arch')" t\Legacy\fail-more.t t\Legacy\More.t
t\Legacy\fail-more.t .. 1/81 #   Failed test at t\Legacy\fail-more.t line 133.
#          got: 'Use of apostrophe as package separator was deprecated in Perl 5.37.9,
# and will be removed in Perl 5.42.0.  You should change code that uses
# Test::More::isn't() to use Test::More::isnt() as a replacement at t\Legacy\fail-more.t line 132
# '
#     expected: 'Use of apostrophe as package separator was deprecated in Perl 5.37.9,
# and will be removed in Perl 5.42.0.  You should change code that uses
# Test::More::isn't() to use Test::More::isnt() as a replacement at t/Legacy/fail-more.t line 132
# '
t\Legacy\fail-more.t .. Failed 1/81 subtests
t\Legacy\More.t ....... 1/57
#   Failed test 'Got expected warning from isn::t() under use warnings'
#   at t\Legacy\More.t line 32.
#          got: 'Use of apostrophe as package separator was deprecated in Perl 5.37.9,
# and will be removed in Perl 5.42.0.  You should change code that uses
# Test::More::isn't() to use Test::More::isnt() as a replacement at t\Legacy\More.t line 31
# '
#     expected: 'Use of apostrophe as package separator was deprecated in Perl 5.37.9,
# and will be removed in Perl 5.42.0.  You should change code that uses
# Test::More::isn't() to use Test::More::isnt() as a replacement at t/Legacy/More.t line 31
# '
# Looks like you failed 1 test of 57.
t\Legacy\More.t ....... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/57 subtests

Test Summary Report
-------------------
t\Legacy\fail-more.t (Wstat: 0 Tests: 81 Failed: 1)
  Failed test:  15
t\Legacy\More.t     (Wstat: 256 (exited 1) Tests: 57 Failed: 1)
  Failed test:  8
  Non-zero exit status: 1
Files=2, Tests=138,  1 wallclock secs ( 0.06 usr +  0.01 sys =  0.08 CPU)
Result: FAIL
Failed 2/2 test programs. 2/138 subtests failed.
NMAKE : fatal error U1077: 'C:\perl64\5.38.0rc1\bin\perl.exe' : return code '0x1'
Stop.

C:\home\mountain5380rc1\Test-Simple-1.302201>

Thank you,

jkeenan commented 2 months ago

Hello,

Since version 1.302193, when I run "nmake test" on Windows, t\Legacy\fail-more.t and t\Legacy\More.t appear to fail. Please see example outputs below.

C:\home\mountain5380rc1\Test-Simple-1.302201>nmake test TEST_FILES="t\Legacy\fail-more.t t\Legacy\More.t"

Microsoft (R) Program Maintenance Utility Version 14.33.31630.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        "C:\perl64\5.38.0rc1\bin\perl.exe" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches;
harness(0, 'blib\lib', 'blib\arch')" t\Legacy\fail-more.t t\Legacy\More.t
t\Legacy\fail-more.t .. 1/81 #   Failed test at t\Legacy\fail-more.t line 133.
#          got: 'Use of apostrophe as package separator was deprecated in Perl 5.37.9,
# and will be removed in Perl 5.42.0.  You should change code that uses
# Test::More::isn't() to use Test::More::isnt() as a replacement at t\Legacy\fail-more.t line 132
# '
#     expected: 'Use of apostrophe as package separator was deprecated in Perl 5.37.9,
# and will be removed in Perl 5.42.0.  You should change code that uses
# Test::More::isn't() to use Test::More::isnt() as a replacement at t/Legacy/fail-more.t line 132
# '
t\Legacy\fail-more.t .. Failed 1/81 subtests
t\Legacy\More.t ....... 1/57
#   Failed test 'Got expected warning from isn::t() under use warnings'
#   at t\Legacy\More.t line 32.
#          got: 'Use of apostrophe as package separator was deprecated in Perl 5.37.9,
# and will be removed in Perl 5.42.0.  You should change code that uses
# Test::More::isn't() to use Test::More::isnt() as a replacement at t\Legacy\More.t line 31
# '
#     expected: 'Use of apostrophe as package separator was deprecated in Perl 5.37.9,
# and will be removed in Perl 5.42.0.  You should change code that uses
# Test::More::isn't() to use Test::More::isnt() as a replacement at t/Legacy/More.t line 31
# '
# Looks like you failed 1 test of 57.
t\Legacy\More.t ....... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/57 subtests

Test Summary Report
-------------------
t\Legacy\fail-more.t (Wstat: 0 Tests: 81 Failed: 1)
  Failed test:  15
t\Legacy\More.t     (Wstat: 256 (exited 1) Tests: 57 Failed: 1)
  Failed test:  8
  Non-zero exit status: 1
Files=2, Tests=138,  1 wallclock secs ( 0.06 usr +  0.01 sys =  0.08 CPU)
Result: FAIL
Failed 2/2 test programs. 2/138 subtests failed.
NMAKE : fatal error U1077: 'C:\perl64\5.38.0rc1\bin\perl.exe' : return code '0x1'
Stop.

C:\home\mountain5380rc1\Test-Simple-1.302201>

My impression is that there are two inter-related problems here. First, some of the test messages have Unix-style forward-slashes as path separators, which may not match test output where Windows prints back-slashes.

Second, Test-Simple-1.302193 aimed at adapting the distribution to the deprecation of apostrophes as package separators and inside variable names. But in the Perl core distribution, we advanced from deprecation to removal of that functionality earlier this month.

commit 0c81a5c9ca0287c0ee954a71b1bffdeb2cb3e9c8
Merge: 3e14b2fcf5 83d4e742eb
Author:     Tony Cook <tony@develop-help.com>
AuthorDate: Sun Aug 11 21:03:07 2024
Commit:     Tony Cook <tony@develop-help.com>
CommitDate: Sun Aug 11 21:03:07 2024

    remove use of ' as a package separator

In the Perl core distribution, we are as of today using Test-Simple-1.302201. Perhaps the t/Legacy need further adaptation -- but I'm not familiar enough with Test-Simple's own development to know how to proceed. @exodist?

exodist commented 2 months ago

I will remove all uses of ' as a package separator and adjust the tests accordingly.

Can you spell out for me which tests use hardcoddd unix path separators? I can fix those too.

twata1 commented 2 months ago

It is thought to be here. https://github.com/Test-More/test-more/blob/5bacf7cdfc561e88fd63fb0526899062911d5b92/t/Legacy/More.t#L35 https://github.com/Test-More/test-more/blob/5bacf7cdfc561e88fd63fb0526899062911d5b92/t/Legacy/fail-more.t#L155

exodist commented 2 months ago

Please check #1003 to see if it fixes the issue

twata1 commented 2 months ago

I can confirm that this issue has been fixed!

The result logs of nmake test and gmake test are attached for your reference. nmake-test-gh1003.log.txt gmake-test-gh1003.log.txt

People who build Perl using Visual Studio must be pleased. I thank you for your correspondence!

exodist commented 2 months ago

Running my usual downstream tests to make sure nothing is broken, then will release to cpan