Behavior of numeric bitwise operators appears to change from Perl 5

[jkeenan]

Leaving use p5; in a program can have a false positive effect on test results. Removing that use p5; way cause errors or warnings, at compile-time or run-time, but at least it lets us know how the program performs under our current implementation of Perl 7 semantics.

Case in point. Consider the following program:

$ cat bitwise.pl 
my $t = 0;

my $Aaz = chr(ord("A") & ord("z"));
my $Aoz = chr(ord("A") | ord("z"));
my $Axz = chr(ord("A") ^ ord("z"));
note($_) for ($Aaz, $Aoz, $Axz);

is (("AAAAA" & "zzzzz"), ($Aaz x 5), 'bitwise and');
is (("AAAAA" | "zzzzz"), ($Aoz x 5), 'bitwise or');
is (("AAAAA" ^ "zzzzz"), ($Axz x 5), 'bitwise xor');

sub note { my $arg = shift; print "# $arg\n"; }
sub is {
    my ($got, $expected, $description) = @_;
    if ($got eq $expected) {
        print "ok ", ++$t, " - $description\n";
    }
    else {
        print "not ok ", ++$t, " - $description\n";
    }
}

Note the absence of either use strict or use warnings from the program.

If I run it as is in Perl 5, I get:

$ perl bitwise.pl 
# @
# {
# ;
ok 1 - bitwise and
ok 2 - bitwise or
ok 3 - bitwise xor

Runs clean. Now suppose I run it with strict enabled.

$ perl -Mstrict bitwise.pl 
# @
# {
# ;
ok 1 - bitwise and
ok 2 - bitwise or
ok 3 - bitwise xor

Again runs clean. Now suppose I run it with warnings enabled.

$ perl -w bitwise.pl 
# @
# {
# ;
ok 1 - bitwise and
ok 2 - bitwise or
ok 3 - bitwise xor

Still runs clean.

Now, suppose I run it with the "Perl 7" executable from the p7 branch, say, at commit 8c5f5218ed6d0fd4eba8e20e2089cb3fcbe35a2c (Jul 19 2020). In this environment, both strict and warnings are on by default.

$ ./perl -Ilib ~/learn/perl/p5p/bitwise.pl 
# @
# {
# ;
Argument "AAAAA" isn't numeric in numeric bitwise and (&) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 8.
Argument "zzzzz" isn't numeric in numeric bitwise and (&) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 8.
not ok 1 - bitwise and
Argument "AAAAA" isn't numeric in numeric bitwise or (|) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 9.
Argument "zzzzz" isn't numeric in numeric bitwise or (|) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 9.
not ok 2 - bitwise or
Argument "AAAAA" isn't numeric in numeric bitwise xor (^) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 10.
Argument "zzzzz" isn't numeric in numeric bitwise xor (^) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 10.
not ok 3 - bitwise xor

Now all unit tests warn -- twice! -- and all unit tests fail.

This is a conundrum. On the one hand, per perldoc perldiag:

Argument "%s" isn't numeric%s
    (W numeric) The indicated string was fed as an argument to an
    operator that expected a numeric value instead. If you're fortunate
    the message will identify which operator was so unfortunate.

So, none of the arguments to any of the 3 numeric bitwise operators were ever numeric; they were always strings. So when I ran this program in Perl 5 with warnings enabled, I believe I should have gotten warnings -- but I did not.

Under Perl 7 semantics, I am getting "non-numeric" warnings as the documentation would suggest I should. But the ultimate behavior of the program has changed: failures instead of successes.

Now, you may ask, what happens if in your is() testing subroutine, you test for numeric equality rather than string equality. That is, what if you add this:

sub numeric_is {
    my ($got, $expected, $description) = @_;
    if ($got == $expected) {
        print "ok ", ++$t, " - $description\n";
    }
    else {
        print "not ok ", ++$t, " - $description\n";
    }
}

... and added 3 unit tests to the body of the program:

numeric_is (("AAAAA" & "zzzzz"), ($Aaz x 5), 'bitwise and');
numeric_is (("AAAAA" | "zzzzz"), ($Aoz x 5), 'bitwise or');
numeric_is (("AAAAA" ^ "zzzzz"), ($Axz x 5), 'bitwise xor');

Now, when I run this under Perl 5, either "raw" or with -Mstrict, I get no change in results.

However, running it under Perl 5 with warnings enabled, the program emits warnings:

$ perl -w bitwise.pl 
# @
# {
# ;
ok 1 - bitwise and
ok 2 - bitwise or
ok 3 - bitwise xor
Argument "@@@@@" isn't numeric in numeric eq (==) at bitwise.pl line 29.
Argument "@@@@@" isn't numeric in numeric eq (==) at bitwise.pl line 29.
ok 4 - bitwise and
Argument "{{{{{" isn't numeric in numeric eq (==) at bitwise.pl line 29.
Argument "{{{{{" isn't numeric in numeric eq (==) at bitwise.pl line 29.
ok 5 - bitwise or
Argument ";;;;;" isn't numeric in numeric eq (==) at bitwise.pl line 29.
Argument ";;;;;" isn't numeric in numeric eq (==) at bitwise.pl line 29.
ok 6 - bitwise xor

But the additional 3 tests still pass!

Now, under Perl 7:

$ ./perl -Ilib ~/learn/perl/p5p/bitwise.pl 
# @
# {
# ;
Argument "AAAAA" isn't numeric in numeric bitwise and (&) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 8.
Argument "zzzzz" isn't numeric in numeric bitwise and (&) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 8.
not ok 1 - bitwise and
Argument "AAAAA" isn't numeric in numeric bitwise or (|) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 9.
Argument "zzzzz" isn't numeric in numeric bitwise or (|) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 9.
not ok 2 - bitwise or
Argument "AAAAA" isn't numeric in numeric bitwise xor (^) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 10.
Argument "zzzzz" isn't numeric in numeric bitwise xor (^) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 10.
not ok 3 - bitwise xor
Argument "AAAAA" isn't numeric in numeric bitwise and (&) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 12.
Argument "zzzzz" isn't numeric in numeric bitwise and (&) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 12.
Argument "@@@@@" isn't numeric in numeric eq (==) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 29.
ok 4 - bitwise and
Argument "AAAAA" isn't numeric in numeric bitwise or (|) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 13.
Argument "zzzzz" isn't numeric in numeric bitwise or (|) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 13.
Argument "{{{{{" isn't numeric in numeric eq (==) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 29.
ok 5 - bitwise or
Argument "AAAAA" isn't numeric in numeric bitwise xor (^) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 14.
Argument "zzzzz" isn't numeric in numeric bitwise xor (^) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 14.
Argument ";;;;;" isn't numeric in numeric eq (==) at /home/jkeenan/learn/perl/p5p/bitwise.pl line 29.
ok 6 - bitwise xor

Warnings in all 6 unit tests -- but when tested with numeric_is() the tests pass.

What's going on here?

(Real world impact: Once I remove use p5 from t/op/bop.t, all hell breaks loose. Given that the Perl 5 behavior may have been wrong all along, I don't know how to adapt tests to Perl 7.)

Thank you very much. Jim Keenan

[rurban]

You need to compare against the added prefix

use feature 'bitwise';
use warnings 'numeric';

Only the feature bitwise converts the bit_and into i_bit_and ops (called nbit_and in perl5), which will be then we warned against. bitwise is really enforcing integer context for the 3 logical compare binops. (n is a misnomer)

Just fix your testcase.