POV-Ray / povray

The Persistence of Vision Raytracer: http://www.povray.org/
GNU Affero General Public License v3.0
1.34k stars 282 forks source link

vfesession.cpp:342 overflow in conversion from long long int to long int on m68k #384

Open baryluk opened 4 years ago

baryluk commented 4 years ago

Summary

Compiling povray, on m68k machine leads to compiler warning

Environment

Unix Build Command Sequence

cd unix ; prebuild.sh ; cd ..
mv libraries libraries.dontuse
./configure --disable-optimiz --disable-strip CFLAGS="-O0 -ggdb" LIBS="-ltiff -lpng -ljpeg -lz -lrt -lm" CXXFLAGS="-O0 -std=c++11 -ggdb" COMPILED_BY="me <b@gz.org>"
make

Unix Configure Output

===============================================================================
Configure POV-Ray version 3.8.0-alpha
===============================================================================

This is an unofficial version compiled by:
 me <b@gz.org>
The POV-Ray Team(tm) is not responsible for supporting this version.

Environment
-----------
checking build system type... m68k-unknown-linux-gnu
checking host system type... m68k-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking whether $C_INCLUDE_PATH contains the "." path... no
checking whether $CPLUS_INCLUDE_PATH contains the "." path... no
checking whether make supports the include directive... yes (GNU style)
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar

Programs
--------
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for stdlib.h... (cached) yes
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking whether the g++ compiler works... yes
checking how to run the C++ preprocessor... g++ -E
checking whether g++ supports C++11 features by default... yes
checking for C++ compiler vendor... gnu
checking for g++ version... 9
checking for ranlib... ranlib
checking for stat format option... --format=

Libraries
---------
checking whether to link with cygwin DLL... no
checking whether to enable static linking... no
checking for a sed that does not truncate output... /usr/bin/sed
checking whether gcc is Clang... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking whether more special flags are required for pthreads... no
checking for PTHREAD_PRIO_INHERIT... yes
checking for boostlib >= 1.38... yes
checking for sin in -lmkl... no
checking for sin in -lm... yes
checking for clock_gettime in -lrt... yes
checking whether to use the ZLIB library... yes
checking for library containing zlibVersion... none required
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking for libz version >= 1.2.1... 1.2.11, ok
checking whether to use the PNG library... yes
checking for library containing png_get_libpng_ver... none required
checking png.h usability... yes
checking png.h presence... yes
checking for png.h... yes
checking for libpng version >= 1.2.5... 1.6.37, ok
checking whether to use the JPEG library... yes
checking for library containing jpeg_std_error... none required
checking jpeglib.h usability... yes
checking jpeglib.h presence... yes
checking for jpeglib.h... yes
checking for libjpeg version >= 6b (62)... 62, ok
checking whether to use the TIFF library... yes
checking for library containing TIFFGetVersion... none required
checking tiffio.h usability... yes
checking tiffio.h presence... yes
checking for tiffio.h... yes
checking for libtiff version >= 3.6.1... 4.0.10, ok
checking whether to use the OpenEXR library... yes
checking for pkg-config... pkg-config
checking for OpenEXR's pkg-config... not found
configure: WARNING: all program features using the OpenEXR library are disabled
checking for X... libraries /usr/lib, headers /usr/include
checking whether -R must be followed by a space... neither works
checking for dnet_ntoa in -ldnet... no
checking for dnet_ntoa in -ldnet_stub... no
checking for gethostbyname... yes
checking for connect... yes
checking for remove... yes
checking for shmat... yes
checking for IceConnectionNumber in -lICE... no
checking X11/Xlib.h usability... no
checking X11/Xlib.h presence... no
checking for X11/Xlib.h... no

Language constructs and functions
---------------------------------
checking whether time.h and sys/time.h may both be included... yes
checking time.h usability... yes
checking time.h presence... yes
checking for time.h... yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking sys/resource.h usability... yes
checking sys/resource.h presence... yes
checking for sys/resource.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking sys/wait.h usability... yes
checking sys/wait.h presence... yes
checking for sys/wait.h... yes
checking for unistd.h... (cached) yes
checking for int8_t... yes
checking for int16_t... yes
checking for int32_t... yes
checking for int64_t... yes
checking for size_t... yes
checking whether struct tm is in sys/time.h or time.h... time.h
checking size of int... 4
checking size of long int... 4
checking size of size_t... 4
checking size of float... 4
checking for working memcmp... yes
checking for getcwd... yes
checking for readlink... yes
checking for nanosleep... yes
checking for usleep... yes
checking for useconds_t... yes
checking for clock_gettime... yes
checking whether CLOCK_MONOTONIC is declared... yes
checking whether CLOCK_REALTIME is declared... yes
checking whether CLOCK_PROCESS_CPUTIME_ID is declared... yes
checking whether CLOCK_THREAD_CPUTIME_ID is declared... yes
checking for clockid_t... yes
checking for getrusage... yes
checking whether RUSAGE_SELF is declared... yes
checking whether RUSAGE_THREAD is declared... yes
checking whether RUSAGE_LWP is declared... yes
checking for gettimeofday... yes

Compiling
---------
checking whether to enable pipes for communications... yes
checking whether g++ accepts -pipe... yes
checking whether g++ accepts -Wno-multichar... yes
checking whether g++ accepts -Wno-write-strings... yes
checking whether g++ accepts -fno-enforce-eh-specs... yes
checking whether g++ accepts -Wno-non-template-friend... yes
checking whether g++ accepts -Wsuggest-override... yes
checking whether to enable I/O restrictions... yes
checking whether to enable debugging... no
checking whether to enable profiling... no
checking whether to enable stripping... no
checking whether to enable optimizations... no

Floating Point Features
-----------------------
checking limits usability... yes
checking limits presence... yes
checking for limits... yes
checking whether NaNs are supported... yes
checking cmath usability... yes
checking cmath presence... yes
checking for cmath... yes
checking whether NaNs can be identified using std::isnan()... yes
checking whether infinite values are supported... yes
checking for cmath... (cached) yes
checking whether infinities can be identified using std::isinf()... no
checking whether infinities can be identified using global isinf()... no
checking whether infinities can be identified by comparison to the maximum value... no

Makefiles
---------
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating source/Makefile
config.status: creating vfe/Makefile
config.status: creating platform/Makefile
config.status: creating unix/Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands

===============================================================================
POV-Ray 3.8.0-alpha has been configured.

Built-in features:
  I/O restrictions:          enabled
  X Window display:          disabled
  Supported image formats:   gif tga iff ppm pgm hdr png jpeg tiff
  Unsupported image formats: openexr

Compilation settings:
  Build architecture:  m68k-unknown-linux-gnu
  Built/Optimized for: m68k-unknown-linux-gnu
  Compiler vendor:     gnu
  Compiler version:    g++ 9
  Compiler flags:      -pipe -Wno-multichar -Wno-write-strings -fno-enforce-eh-specs -Wno-non-template-friend -Wsuggest-override -O0 -std=c++11 -ggdb -pthread
  Libraries:           -lrt -lm -ltiff -lpng -ljpeg -lz -lrt -lm -pthread 

Type 'make check' to build the program and run a test render.
Type 'make install' to install POV-Ray on your system.

The POV-Ray components will be installed in the following directories:
  Program (executable):       /usr/local/bin
  System configuration files: /usr/local/etc/povray/3.8
  User configuration files:   /root/.povray/3.8
  Standard include files:     /usr/local/share/povray-3.8/include
  Standard INI files:         /usr/local/share/povray-3.8/ini
  Standard demo scene files:  /usr/local/share/povray-3.8/scenes
  Documentation (text, HTML): /usr/local/share/doc/povray-3.8
  Unix man page:              /usr/local/share/man
===============================================================================

root@debian:~/povray# make clean

Steps to Reproduce

  1. You can follow guide at https://wiki.debian.org/M68k/QemuSystemM68k to obtain m68k emulator and working system.
  2. Run the machine, install minimal build depndencies
  3. Try to compile povray using above commit, and options.

Another option:

  1. On amd64 Debian system install g++-9-m68k-linux-gnu and binutils-m68k-linux-gnu , and then cross-compile the povray with appropriate extra configure flags.

The exact versions of libraries (like boost) used doesn't really matter here, as can be seen below.

These are main dependencies I used:

# dpkg -l | egrep 'boost|png|jpeg|tiff|zlib' | grep dev | awk '{print $2, $3;}'
libboost-atomic1.67-dev:m68k 1.67.0-10
libboost-chrono1.67-dev:m68k 1.67.0-10
libboost-date-time-dev:m68k 1.67.0.2
libboost-date-time1.67-dev:m68k 1.67.0-10
libboost-dev:m68k 1.67.0.2
libboost-serialization1.67-dev:m68k 1.67.0-10
libboost-system1.67-dev:m68k 1.67.0-10
libboost-thread-dev:m68k 1.67.0.2
libboost-thread1.67-dev:m68k 1.67.0-10
libboost1.67-dev:m68k 1.67.0-10
libjpeg-dev 1:1.5.2-2
libjpeg62-turbo-dev:m68k 1:1.5.2-2+b1
libpng-dev:m68k 1.6.37-1
libtiff-dev:m68k 4.0.10+git191003-1
libtiff5-dev 4.0.10+git191003-1
zlib1g-dev:m68k 1:1.2.11.dfsg-1
# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/m68k-linux-gnu/9/lto-wrapper
Target: m68k-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 9.2.1-12' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,c++,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=m68k-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libssp --disable-libitm --disable-libsanitizer --disable-libquadmath --disable-libquadmath-support --enable-plugin --with-system-zlib --disable-libphobos --enable-multiarch --disable-werror --disable-werror --disable-multilib --enable-checking=release --build=m68k-linux-gnu --host=m68k-linux-gnu --target=m68k-linux-gnu
Thread model: posix
gcc version 9.2.1 20191022 (Debian 9.2.1-12) 
#

Expected Behavior

No integer overflows of literals.

Actual Behavior

g++ -DHAVE_CONFIG_H -DPOVLIBDIR=\"/usr/local/share/povray-3.8\" -DPOVCONFDIR=\"/usr/local/etc/povray/3.8\" -DPOVCONFDIR_BACKWARD=\"/usr/local/etc\" -I. -I..  -I../unix/povconfig -I../platform/unix  -I../vfe/unix -I../unix -I../source   -pipe -Wno-multichar -Wno-write-strings -fno-enforce-eh-specs -Wno-non-template-friend -Wsuggest-override -O0 -std=c++11 -ggdb -pthread -MT vfesession.o -MD -MP -MF $depbase.Tpo -c -o vfesession.o vfesession.cpp &&\
mv -f $depbase.Tpo $depbase.Po
vfesession.cpp: In member function ‘virtual bool vfe::vfeSession::GetNextCombinedMessage(vfe::vfeSession::MessageType&, std::string&)’:
vfesession.cpp:342:40: warning: overflow in conversion from ‘long long int’ to ‘long int’ changes value from ‘9223372036854775807’ to ‘-1’ [-Woverflow]
  342 |   POV_LONG                    mqTime = 0x7fffffffffffffffLL ;
      |                                        ^~~~~~~~~~~~~~~~~~~~
vfesession.cpp:343:40: warning: overflow in conversion from ‘long long int’ to ‘long int’ changes value from ‘9223372036854775807’ to ‘-1’ [-Woverflow]
  343 |   POV_LONG                    sqTime = 0x7fffffffffffffffLL ;
      |                                        ^~~~~~~~~~~~~~~~~~~~
vfesession.cpp:344:40: warning: overflow in conversion from ‘long long int’ to ‘long int’ changes value from ‘9223372036854775807’ to ‘-1’ [-Woverflow]
  344 |   POV_LONG                    cqTime = 0x7fffffffffffffffLL ;
      |                                        ^~~~~~~~~~~~~~~~~~~~
vfesession.cpp: In member function ‘virtual bool vfe::vfeSession::GetNextNonStatusMessage(vfe::vfeSession::MessageType&, std::string&, pov_base::UCS2String&, int&, int&)’:
vfesession.cpp:389:40: warning: overflow in conversion from ‘long long int’ to ‘long int’ changes value from ‘9223372036854775807’ to ‘-1’ [-Woverflow]
  389 |   POV_LONG                    mqTime = 0x7fffffffffffffffLL ;
      |                                        ^~~~~~~~~~~~~~~~~~~~
vfesession.cpp:390:40: warning: overflow in conversion from ‘long long int’ to ‘long int’ changes value from ‘9223372036854775807’ to ‘-1’ [-Woverflow]
  390 |   POV_LONG                    cqTime = 0x7fffffffffffffffLL ;
      |                                        ^~~~~~~~~~~~~~~~~~~~

Suggested Solution

POV_LONG type change? Preprocessing section for different integer widths?

./configure detecting properly sizes of integer types?

baryluk commented 4 years ago

BTW. I don't get this warning when compiling vfesession.o with povray latest-stable aka 3.7.0.8 on the same machine and using same toolchain and flags. So it looks like a regression.

baryluk commented 4 years ago

Yes, it looks like the issue is that in 3.7.0.8 the POV_LONG is defined uncondiotnally as long long on POSIX.

But now it is instead long long on 32-bit platforms, to make it 64-bit; and long on 64-bit platforms, to make it not 128-bit, and also 64-bit.

This happens in ./unix/povconfig/syspovconfig_posix.h , unix/povconfig/syspovconfig_gnu.h and in unix/povconfig/syspovconfig_bsd.h and in unix/povconfig/syspovconfig_osx.h in similar way. In fact all these files are identical, and it is a bit silly to even have them and cluster source code.

#if defined(_POSIX_V6_LPBIG_OFFBIG) || defined(_POSIX_V6_LP64_OFF64)
    // long is at least 64 bits.
    #define POV_LONG long
#elif defined(_POSIX_V6_ILP32_OFFBIG) || defined(_POSIX_V6_ILP32_OFF32)
    // long is 32 bits.
    #define POV_LONG long long
#else
    // Unable to detect long size at compile-time, assuming less than 64 bits.
    #define POV_LONG long long
#endif

The problem is that this is not correct on few levels.

First: long long can be bigger than 64-bits on 32-bit platforms. In fact even long can be bigger than 64-bit on 64-bit platforms. In theory at least.

Second: You use these macros incorrectly, it should be:

#if (defined(_POSIX_V7_LPBIG_OFFBIG) && (_POSIX_V7_LPBIG_OFFBIG == 1)) || (defined(_POSIX_V7_LP64_OFF32) && (_POSIX_V7_LP64_OFF32 == 1))
    // long is at exactly 64 bits.
    #define POV_LONG long
#elif (defined(_POSIX_V_7_ILP32_OFFBIG) && (_POSIX_V_7_ILP32_OFFBIG == 1)) || (defined(_POSIX_V7_ILP32_OFF32) && (_POSIX_V7_ILP32_OFF32 == 1))
    // long is exactly 32 bits, but long long is guaranteed to be at least 64-bit.
    // lets hope it is not bigger (it probably isn't).
    #define POV_LONG long long
#else
    // Unable to detect long size at compile-time, assuming less than 64 bits.
    #define POV_LONG long long
#endif

Additional check against 1 is needed, as these macros can be defined by equal -1 to indicate lack of support.

For example on m68k-linux-gnu triplet:

/usr/include/m68k-linux-gnu/bits/environments.h:

#if __WORDSIZE == 64

(snip)

#else /* __WORDSIZE == 32 */

/* By default we have 32-bit wide `int', `long int', pointers and `off_t'
   and all platforms support LFS.  */
# define _POSIX_V7_ILP32_OFF32  1
# define _POSIX_V7_ILP32_OFFBIG 1
# define _POSIX_V6_ILP32_OFF32  1
# define _POSIX_V6_ILP32_OFFBIG 1
# define _XBS5_ILP32_OFF32  1
# define _XBS5_ILP32_OFFBIG 1

/* We optionally provide an environment with the above size but an 64-bit
   side `off_t'.  Therefore we don't define _POSIX_V7_ILP32_OFFBIG.  */

/* We can never provide environments with 64-bit wide pointers.  */
# define _POSIX_V7_LP64_OFF64   -1
# define _POSIX_V7_LPBIG_OFFBIG -1
# define _POSIX_V6_LP64_OFF64   -1
# define _POSIX_V6_LPBIG_OFFBIG -1
# define _XBS5_LP64_OFF64   -1
# define _XBS5_LPBIG_OFFBIG -1

/* CFLAGS.  */
#define __ILP32_OFFBIG_CFLAGS   "-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"

#endif /* __WORDSIZE == 32 */

This is a standard compliant behaviour and not a bug in gcc or glibc.

Third: Even that fixed macros is not 100% bullet proof, due to long long possibly being bigger than 64-bits even on 32-bit platforms. In general it shouldn't be a big issue, but in fact it can.

Third: It is all unnecessary. You are already including proper POSIX headers that automatically provide necessary types. Namely uint32_t, uint64_r and int64_t. Considering you need C++11 and other reasonably modern libraries and compilers, really it will always be available on POSIX when compiling povray.

Just use #define POV_LONG int64_t, it should work fine.

#include <stdint.h> in C (since C99), and #include <cstdint> in C++ (since C++11). It is probably also fine to include the C one in C++.

Over even better replace all uses of POV_LONG in the entire source try with int64_t if possible. Similar with POV_ULONG to uint64_t. For platforms that don't provide that types, define them using some macros just for these platforms.

PS. In Debian there is even a patch to detect the incorrect detection of POV_LONG at runtime.

https://salsa.debian.org/debian/povray/blob/master/debian/patches/verify_POV_LONG_is_64bit.patch

I highly recommend to simply remove POV_LONG and use int64_t.

c-lipka commented 3 years ago

Those files date back to a time when C++11 was still a new thing, and POV-Ray tried to avoid requiring it, to be compatible with older compilers. We simply haven't found the time and energy yet to fully leverage C++11 features all throughout our code.

As for int64_t, that is not an option, as it is optional (no pun intended) according to the C++11 and later standards. Or, more specifically, it is either mandatory or prohibited, depending on the details of the hardware architecture. Instead, int_least64_t is the proper type to use.

c-lipka commented 3 years ago

For the records: As this applies to an exotic platform, we probably don't want to tackle it in the upcoming v3.8.0 release. It does limit the set of compatible platforms though, so we probably do want to address it in a later v3.8 maintenance release.