perl's NaN is -NaN

[t-a-k]

Description NaN values generated by perl, such as the result of a string-to-number conversion or the returned value of POSIX::NAN, are actually -NaN (i.e. its sign bit is inverted).

As -NaN is also NaN, this seems not to be a serious issue, but inconsistent with underlying C library/compilers.

Steps to Reproduce

$ perl -wle 'print unpack "H*", pack "F>", 0 + "NaN"'
fff8000000000000
$ perl -MPOSIX -wle 'print "NAN is negative" if POSIX::signbit(POSIX::NAN())'
NAN is negative

Expected behavior perl should generate NaNs with their sign bits cleared.

Perl configuration

$ perl -V
Summary of my perl5 (revision 5 version 33 subversion 6) configuration:
  Commit id: a55134cc382529aee24f81156d7540b61a753544
  Platform:
    osname=linux
    osvers=4.19.0-13-686-pae
    archname=i686-linux
    uname='linux x41 4.19.0-13-686-pae #1 smp debian 4.19.160-2 (2020-11-28) i686 gnulinux '
    config_args='-de -Dusedevel -DEBUGGING'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=undef
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O2 -g'
    cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='8.3.0'
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=4
    doublesize=8
    byteorder=1234
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=12
    longdblkind=3
    ivtype='long'
    ivsize=4
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=4
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/i686-linux-gnu/8/include-fixed /usr/include/i386-linux-gnu /usr/lib /lib/i386-linux-gnu /lib/../lib /usr/lib/i386-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64
    libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.28.so
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.28'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl): 
  Compile-time options:
    DEBUGGING
    HAS_TIMES
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_DEVEL
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
  Built under linux
  Compiled at Jan 14 2021 01:29:14
  @INC:
    lib
    /usr/local/lib/perl5/site_perl/5.33.6/i686-linux
    /usr/local/lib/perl5/site_perl/5.33.6
    /usr/local/lib/perl5/5.33.6/i686-linux
    /usr/local/lib/perl5/5.33.6

[sisyphus]

I'm not seeing the same on my Windows 32-bit build of 5.33.6 (up to commit 80ac1a60). Could it be a libc thing ?

C:\>perl -wle "print unpack 'H*', pack 'F>', 0 + 'NaN'"
7ff8000000000000

C:\>perl -MPOSIX -wle "print 'NAN is negative' if POSIX::signbit(POSIX::NAN())"

C:\>

Here's my perl -V Don't be put off by the various occurrences of "5.33.9". It is still only 5.33.6

Summary of my perl5 (revision 5 version 33 subversion 6) configuration:

  Platform:
    osname=MSWin32
    osvers=6.1.7601
    archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended
    useposix=true
    d_sigaction=undef
    useithreads=define
    usemultiplicity=define
    use64bitint=undef
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='gcc'
    ccflags =' -DWIN32 -fdiagnostics-color=never -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -D__USE_MINGW_ANSI_STDIO -fwrapv -fno-strict-aliasing -mms-bitfields'
    optimize='-s -O2'
    cppflags='-DWIN32'
    ccversion=''
    gccversion='8.3.0'
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=4
    doublesize=8
    byteorder=1234
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=12
    longdblkind=3
    ivtype='long'
    ivsize=4
    nvtype='double'
    nvsize=8
    Off_t='long long'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='g++'
    ldflags ='-s -L"C:\perl-5.33.9\lib\MSWin32-x86-multi-thread\CORE" -L"C:\_32\gcc-straw-830\mingw32\lib"'
    libpth=C:\_32\gcc-straw-830\mingw32\lib C:\_32\gcc-straw-830\mingw32\i686-w64-mingw32\lib C:\_32\msys_830\1.0\local\lib C:\_32\gcc-straw-830\mingw32\lib\gcc\i686-w64-mingw32\8.3.0
    libs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    perllibs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    libc=
    so=dll
    useshrplib=true
    libperl=libperl533.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs
    dlext=dll
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags='-mdll -s -L"C:\perl-5.33.9\lib\MSWin32-x86-multi-thread\CORE" -L"C:\_32\gcc-straw-830\mingw32\lib"'

Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    HAVE_INTERP_INTERN
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_IMPLICIT_CONTEXT
    PERL_IMPLICIT_SYS
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
  Built under MSWin32
  Compiled at Jan 10 2021 11:25:45
  @INC:
    C:/perl-5.33.9/site/lib/MSWin32-x86-multi-thread
    C:/perl-5.33.9/site/lib
    C:/perl-5.33.9/lib/MSWin32-x86-multi-thread
    C:/perl-5.33.9/lib

[jgamble]

I'm getting the same non-result in 64-bit Windows:

C:\users\jgamb> perl -wle "print unpack 'H*', pack 'F>', 0 + 'NaN'"
7ff8000000000000
C:\users\jgamb> perl -MPOSIX -wle "print 'NaN is negative' if POSIX::signbit(POSIX::NAN())"
C:\users\jgamb>

The output of perl -V is

C:\users\jgamb> perl -V
Summary of my perl5 (revision 5 version 32 subversion 0) configuration:

  Platform:
    osname=MSWin32
    osvers=10.0.19041.388
    archname=MSWin32-x64-multi-thread
    uname='Win32 strawberry-perl 5.32.0.1 #1 Sun Aug  2 19:46:04 2020 x64'
    config_args='undef'
    hint=recommended
    useposix=true
    d_sigaction=undef
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='gcc'
    ccflags =' -DWIN32 -DWIN64 -D__USE_MINGW_ANSI_STDIO -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -fwrapv -fno-strict-aliasing -mms-bitfields'
    optimize='-s -O2'
    cppflags='-DWIN32'
    ccversion=''
    gccversion='8.3.0'
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='long long'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='g++'
    ldflags ='-s -L"C:\STRAWB~1\perl\lib\CORE" -L"C:\STRAWB~1\c\lib"'
    libpth=C:\STRAWB~1\c\lib C:\STRAWB~1\c\x86_64-w64-mingw32\lib C:\STRAWB~1\c\lib\gcc\x86_64-w64-mingw32\8.3.0
    libs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    perllibs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    libc=
    so=dll
    useshrplib=true
    libperl=libperl532.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs
    dlext=xs.dll
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags='-mdll -s -L"C:\STRAWB~1\perl\lib\CORE" -L"C:\STRAWB~1\c\lib"'

Characteristics of this binary (from libperl):
  Compile-time options:
    HAS_TIMES
    HAVE_INTERP_INTERN
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_IMPLICIT_CONTEXT
    PERL_IMPLICIT_SYS
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
  Built under MSWin32
  Compiled at Aug  2 2020 19:51:20
  @INC:
    C:/Strawberry/perl/site/lib
    C:/Strawberry/perl/vendor/lib
    C:/Strawberry/perl/lib

[t-a-k]

I'm not seeing the same on my Windows 32-bit build of 5.33.6 (up to commit 80ac1a6).

I'm getting the same non-result in 64-bit Windows:

These seem to be because Windows build uses its own NV_NAN definition in win32.h, which hardcodes NaN as 0x7FF8000000000000.

On non-Windows build, NV_NAN (or PL_nan) seems to come from sqrt(-1.0) in Configure script, but on x86 it seems to return -NaN. I think defining NV_NAN as nan("") or NAN might help, but unfortunately these are not available in C89.

[sisyphus]

There's apparently still some inconsistency in the way that "nan" strings are evaluated when used in numeric context. On Ubuntu :

$ perl  -le 'print unpack "H*", pack "F>", "NaN";'
7ff8000000000000
$ perl -le 'print unpack "H*", pack "F>", 0 + "NaN";'
fff8000000000000

Similarly, on Windows :

C:\>perl -wle "print unpack 'H*', pack 'F>', 0 + '-NaN'"
7ff8000000000000

C:\>perl -wle "print unpack 'H*', pack 'F>', '-NaN'"
fff8000000000000

Consistency would be nice ... but I don't know if that can easily be achieved.

[karenetheridge]

one more data point:

: [ether@aquavit]$; perl -wle "print unpack 'H*', pack 'F>', 0 + 'NaN'"
fff8000000000000
: [ether@aquavit]$; perl -MPOSIX -wle "print 'NaN is negative' if POSIX::signbit(POSIX::NAN())"
NaN is negative

: [ether@aquavit]$; uname -a
Darwin aquavit 17.7.0 Darwin Kernel Version 17.7.0: Mon Aug 31 22:11:23 PDT 2020; root:xnu-4570.71.82.6~1/RELEASE_X86_64 x86_64
: [ether@aquavit]$; sw_vers
ProductName:    Mac OS X
ProductVersion: 10.13.6
BuildVersion:   17G14033
: [ether@aquavit git/conch]$; perl -v

This is perl 5, version 26, subversion 3 (v5.26.3) built for darwin-2level
(with 1 registered patch, see perl -V for more detail)

Copyright 1987-2018, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

[sisyphus]

@t-a-k do you know where POSIX::NAN gets its value from ? I don't think POSIX::NAN should ever be taking its value from perl ... should it ?

And yet, on Ubuntu, I also find that POSIX::NAN reports fff8000000000000. However, both strtod() and the NAN symbol defined in math.h are reporting 7ff8000000000000.

#include <stdio.h>
#include <math.h>
#include <stdlib.h>

int main(void) {
  void *p1, *p2;
  double d1 = NAN; /* from math.h */
  double d2 = strtod("NaN", NULL);
  int i;
  p1 = &d1;
  p2 = &d2;

  for (i = 7; i >= 0; i--) {
    printf("%02x", ((unsigned char*)p1)[i]);
  }
  printf("\n");
  for (i = 7; i >= 0; i--) {
    printf("%02x", ((unsigned char*)p2)[i]);
  }
  printf("\n");
  return 0;
}

At least POSIX::strtod('NAN') is correctly reporting 7ff8000000000000

[t-a-k]

@t-a-k do you know where POSIX::NAN gets its value from ?

@sisyphus I think this is it. @names is fed to ExtUtils::Constant::WriteConstants afterwards.

https://github.com/Perl/perl5/blob/68ad61f5afbf1b95e3717e314a9204b5689471c7/ext/POSIX/Makefile.PL#L106-L110

POSIX::nan() and POSIX::setpayload() seem to have a similar issue, as currently they also get NaN from NV_NAN.

[sisyphus]

Yes - it looks that way. (On Windows, NV_NAN doesn't have the sign bit set; on Ubuntu/Debian NV_NAN does have the sign bit set.) This raises 2 questions, and I don't have answers for either of them : 1) Why does NV_NAN have the sign bit set on some systems, but not others ? 2) Why does POSIX::NAN set its value from NV_NAN (as opposed to, eg, C's strtod('nan', NULL) or math.h's NAN) ?

I personally try to stay removed from considerations regarding the signs of NaNs. (Why should something that is Not a Number even have a sign ?) To me, a NaN is a NaN, and the only difference that I ascribe to NaNs of different sign is that one has the sign bit set, and the other doesn't. Also, I take no interest in payloads.

However, I do think it would be preferable if the sign bit of NV_NAN/POSIX::NAN/POSIX::nan() was consistent across all platforms .... unless, of course, there's some compelling reason to the contrary.

I find it amusing that, on Ubuntu/Debian, POSIX::NAN has the sign bit set, and -POSIX::NAN doesn't.

[Leont]

Why does NV_NAN have the sign bit set on some systems, but not others ?

Because it's using a canned value on some operating systems, and tries to detect it on others.

Why does POSIX::NAN set its value from NV_NAN (as opposed to, eg, C's strtod('nan', NULL) or math.h's NAN) ?

Because the detected value is presumed to be the most helpful value.

[sisyphus]

Is the "most helpful value" the one that the POSIX module should be using ? I have this (perhaps naive) notion that it should be selecting the value that conforms with the POSIX standard.

Now, if it turns out that the POSIX standard doesn't care about where the value comes from, then I think that selecting "the most helpful value" is fine. But the googling that I did suggested that the standard expects the value to be taken from math.h.

[Leont]

Now, if it turns out that the POSIX standard doesn't care about where the value comes from, then I think that selecting "the most helpful value" is fine.

C89 didn't even assume a NAN existed (because some architectures use non-IEEE754 floating point arithmetic).

I'm not sure if the binary value of NaN matters at all TBH. It doesn't to perl, but I can imagine it matters in serialized form.

[sisyphus]

Maybe I misunderstand the purpose of the POSIX module. I've always thought of it as a means to seeing what the underlying compiler/libc thinks. Just as I expect POSIX::strtod() to use the C implementation of strtod() instead of Perl_strtod(), I expect POSIX::NAN to return C's NAN, not perl's NV_NAN.

And, if the underlying compiler/libc does not define the symbol NAN (which would surely be a rare situation) then POSIX::NAN should croak.

So ... apparently POSIX::NAN is looking in (what I believe to be) the inappropriate place, and that's what I'm questioning.

That there's a discrepancy in the sign bit is not so much of an issue. However, it would be nice if every string matching /^nan/i were to numify to a NaN with unset sign bit && every string matching /^-nan/i were to numify to a NaN with a set sign bit - if only to avoid having discussions like this one ;-).

[t-a-k]

I've always thought of it as a means to seeing what the underlying compiler/libc thinks. Just as I expect POSIX::strtod() to use the C implementation of strtod() instead of Perl_strtod(), I expect POSIX::NAN to return C's NAN, not perl's NV_NAN.

I feel the same way, as POSIX.pod describes POSIX module as an interface to POSIX, rather than a (re)implementation of it.

Apart from the issue of the sign bit, I'll try to create a pull request to address the POSIX::NAN issue as a proposal.

[khwilliamson]

On 1/18/21 8:56 AM, TAKAI Kousuke wrote:

I've always thought of it as a means to seeing what the underlying
compiler/libc thinks.
Just as I expect POSIX::strtod() to use the C implementation of
strtod() instead of Perl_strtod(), I expect POSIX::NAN to return C's
NAN, not perl's NV_NAN.

I feel the same way, as POSIX.pod describes POSIX module as an /interface/ to POSIX, rather than a (re)implementation of it.

My view is somewhat different, and I've made lots of changes to this module based on that view, and no one has complained. I view this module as a way for perl code to gain access to functionality not otherwise furnished by the language. The functions in it should be light weight wrappers around the libc calls and which hide the gotchas those calls have (if any) when interacting with perl.

For example, it is hard to get localeconv() to be thread safe, and perl code shouldn't even have to care about that.

There will soon be 4 (and a half) different implementations of setlocale() depending on threading and the underlying platform's capabilities; all of which are transparent to the caller.

I removed a bunch of functions some releases ago that didn't add anything to the perl capabilities. No one complained.

As to why there is some duplicate functionality, I don't know. Maybe there's a different philosophy for them; or maybe they were added before Perl had an adequate substitute.\

Another example, is there is enough functionality exposed in the module so that someone who really wants to deal with a legacy multi-byte locale (such as various ones for East Asian languages) can roll their own handler. The code in POSIX represents a uniform interface that avoids races, sometimes with mutexes, and sometimes by using a reentrant function if available on the platform, all transparent to the caller.

Apart from the issue of the sign bit, I'll try to create a pull request to address the POSIX::NAN issue as a proposal.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Perl/perl5/issues/18473#issuecomment-762335444, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA2DH4RHSAY2PBUDIDNUJ3S2RK2TANCNFSM4WBGHSVQ.

[Leont]

Maybe I misunderstand the purpose of the POSIX module. I've always thought of it as a means to seeing what the underlying compiler/libc thinks.

AFAIK, NAN is not defined in POSIX anyway (or any other standard)

[t-a-k]

AFAIK, NAN is not defined in POSIX anyway (or any other standard)

According to man pages, NAN (and INFINITY) are defined in C99 <math.h>, and in POSIX (IEEE Std 1003.1-2008).

[t-a-k]

Anyway, what I wanted to say is that if the platform defines the C99/POSIX NAN symbol, Perl's NaN (NV_NAN and POSIX::NAN) would be better to be consistent with it. (If the platform doesn't define NAN. I think it is still reasonable to use detected or canned value.)

[Leont]

The current logic was introduced by 44521f3a1782026b7d25cc55af459c3e28cc9bdd and 9e76e8dd3a4eeeececa2572dad4eb11d8526e286, before that time it would simply use NAN.

[t-a-k]

The current logic was introduced by 44521f3 and 9e76e8d, before that time it would simply use NAN.

Thank you for pointing that out. However, I couldn't find any comments or commit logs that refer to the reason why it avoids NAN even on platforms that defines it in <math.h>. Are there any platforms where NAN is unusable?

[Leont]

However, I couldn't find any comments or commit logs that refer to the reason why it avoids NAN even on platforms that defines it in . Are there any platforms where NAN is unusable?

I guess you should ask @jhi

[t-a-k]

I guess you should ask @jhi

Thank you for your advice.

@jhi Would you please comment on this issue?

[sisyphus]

@t-a-k I think (unchecked) that these changes you are looking at were made by @jhi at the time that he was working on the ":nan_payload" stuff in the POSIX module. So I suspect they could be related to that work. I don't know if such vague and possibly irrelevant/incorrect information is of any use to you at all - but I mention it in the off-chance that it might be helpful.

On Tue, Feb 2, 2021 at 12:45 AM TAKAI Kousuke notifications@github.com wrote:

I guess you should ask @jhi https://github.com/jhi

Thank you for your advice.

@jhi https://github.com/jhi Would you please comment on this issue?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Perl/perl5/issues/18473#issuecomment-770867414, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAR3PGKBGP66INMJKISYHTS42V7BANCNFSM4WBGHSVQ .