[PATCH] Revert "utf8.h, utfebcdic.h: Add some assertions"

p5pRT commented 9 years ago

Migrated from rt.perl.org#126045 (status was 'resolved')

Searchable as RT126045$

p5pRT commented 9 years ago

From @bulk88

Created by @bulk88

See attached patch.

before ---------------------------------------- C:\perl521\src\win32>dmake test CCTYPE=MSVC70 CFG=Debug if not exist ".\mini" mkdir ".\mini" if exist config.h del /f config.h copy config_H.vc config.h 1 file(s) copied. rem. > .\mini\.exists cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\toke.obj -Fdmini\toke.pdb ..\toke.c toke.c ..\toke.c(3118) : warning C4244: 'function' : conversion from 'IV' to 'const U8' \, possible loss of data ..\toke.c(3137) : warning C4244: 'function' : conversion from 'IV' to 'const U8' \, possible loss of data ..\toke.c(3660) : warning C4018: '>' : signed/unsigned mismatch cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regcomp.obj -Fdmini\regcomp.pdb ..\regcomp.c regcomp.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regexec.obj -Fdmini\regexec.pdb ..\regexec.c regexec.c ..\regexec.c(2006) : fatal error C1001: INTERNAL COMPILER ERROR (compiler file 'msc1.cpp'\, line 2708) Please choose the Technical Support command on the Visual C++ Help menu\, or open the Technical Support help file for more information

dmake: Error code 130\, while making 'mini\regexec.obj' C:\perl521\src\win32> ------------------------------------------- after ------------------------------------------- C:\perl521\src\win32>dmake test CCTYPE=MSVC70 CFG=Debug if not exist ".\mini" mkdir ".\mini" if exist config.h del /f config.h copy config_H.vc config.h 1 file(s) copied. rem. > .\mini\.exists cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\toke.obj -Fdmini\toke.pdb ..\toke.c toke.c ..\toke.c(3118) : warning C4244: 'function' : conversion from 'IV' to 'const U8' \, possible loss of data ..\toke.c(3137) : warning C4244: 'function' : conversion from 'IV' to 'const U8' \, possible loss of data ..\toke.c(3660) : warning C4018: '>' : signed/unsigned mismatch cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regcomp.obj -Fdmini\regcomp.pdb ..\regcomp.c regcomp.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regexec.obj -Fdmini\regexec.pdb ..\regexec.c regexec.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\op.obj -Fdmini\op.pdb ..\op.c op.c op.c(8397) : warning C4244: 'function' : conversion from 'cv_flags_t' to 'char'\, possible loss of data cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\sv.obj -Fdmini\sv.pdb ..\sv.c sv.c Caught SIGINT. Trying to quit ... dmake: C:\perl521\src\win32> -------------------------------------------

Perl Info

``` Flags: category=core severity=low Site configuration information for perl 5.23.3: Configured by Owner at Sat Sep 12 17:01:50 2015. Summary of my perl5 (revision 5 version 23 subversion 3) configuration: undef undef Platform: osname=MSWin32, osvers=5.1, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef useithreads=define, usemultiplicity=define use64bitint=undef, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cl', ccflags ='-nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32 -D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS', optimize='-O1 -MD -Zi -DNDEBUG -GL', cppflags='-DWIN32' ccversion='14.00.50727.42', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234, doublekind=3 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8, longdblkind=0 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"c:\perl\lib\CORE" -machine:x86 "/manifestdependency:type='Win32' name='Microsoft.Windows.Common-Controls' version='6.0.0.0' processorArchitecture='*' publicKeyToken='6595b64144ccf1df' language='*'"' libpth=\lib libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl523.lib gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"c:\perl\lib\CORE" -machine:x86 "/manifestdependency:type='Win32' name='Microsoft.Windows.Common-Controls' version='6.0.0.0' processorArchitecture='*' publicKeyToken='6595b64144ccf1df' language='*'"' @INC for perl 5.23.3: C:\perl521\src\lib C:\perl521\src\cpan\AutoLoader\lib C:\perl521\src\dist\Carp\lib C:\perl521\src\dist\PathTools C:\perl521\src\dist\PathTools\lib C:\perl521\src\cpan\ExtUtils-Command\lib C:\perl521\src\cpan\ExtUtils-Install\lib C:\perl521\src\cpan\ExtUtils-MakeMaker\lib C:\perl521\src\cpan\ExtUtils-Manifest\lib C:\perl521\src\cpan\File-Path\lib C:\perl521\src\ext\re C:\perl521\src\dist\Term-ReadLine\lib C:\perl521\src\dist\Exporter\lib C:\perl521\src\ext\File-Find\lib C:\perl521\src\cpan\Text-Tabs\lib C:\perl521\src\dist\constant\lib C:\perl521\src\cpan\Text-ParseWords\lib C:\perl521\src\dist\ExtUtils-ParseXS\lib C:\perl521\src\cpan\Getopt-Long\lib C:\perl521\src\cpan\parent\lib C:\perl521\src\cpan\ExtUtils-Constant\lib . Environment for perl 5.23.3: HOME (unset) LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=C:\perl521\bin;C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\IDE;C:\Program Files\Microsoft Visual Studio .NET 2003\VC7\BIN;C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools;C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools\bin\prerelease;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\system32\wbem; PERL_BADLANG (unset) SHELL (unset) ```

p5pRT commented 9 years ago

From @bulk88

0001-Revert-utf8.h-utfebcdic.h-Add-some-assertions.patch

```diff From af3851c8d46d25f5ab5ab6b9a595bdbdb890a92a Mon Sep 17 00:00:00 2001 From: Daniel Dragan Date: Sat, 12 Sep 2015 20:15:23 -0400 Subject: [PATCH] Revert "utf8.h, utfebcdic.h: Add some assertions" This reverts commit e9b19ab7df3480e8f710ca6faad519f6fccdb081. This fixes a Win32 Visual C 2003 DEBUGGING build failure in compiling regexec.obj. cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regexec.obj -Fdmini\regexec.pdb ..\regexec.c regexec.c ..\regexec.c(2006) : fatal error C1001: INTERNAL COMPILER ERROR (compiler file 'msc1.cpp', line 2708) Please choose the Technical Support command on the Visual C++ Help menu, or open the Technical Support help file for more information No response from original author of the code at http://www.nntp.perl.org/group/perl.perl5.porters/2015/09/msg230842.html --- utf8.h | 8 ++++---- utfebcdic.h | 10 ++++------ 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/utf8.h b/utf8.h index 17f0e82..668626f 100644 --- a/utf8.h +++ b/utf8.h @@ -127,8 +127,8 @@ END_EXTERN_C /* Native character to/from iso-8859-1. Are the identity functions on ASCII * platforms */ -#define NATIVE_TO_LATIN1(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) (ch)) -#define LATIN1_TO_NATIVE(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) (ch)) +#define NATIVE_TO_LATIN1(ch) (ch) +#define LATIN1_TO_NATIVE(ch) (ch) /* I8 is an intermediate version of UTF-8 used only in UTF-EBCDIC. We thus * consider it to be identical to UTF-8 on ASCII platforms. Strictly speaking @@ -136,8 +136,8 @@ END_EXTERN_C * because they are 8-bit encodings that serve the same purpose in Perl, and * rarely do we need to distinguish them. The term "NATIVE_UTF8" applies to * whichever one is applicable on the current platform */ -#define NATIVE_UTF8_TO_I8(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) (ch)) -#define I8_TO_NATIVE_UTF8(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) (ch)) +#define NATIVE_UTF8_TO_I8(ch) (ch) +#define I8_TO_NATIVE_UTF8(ch) (ch) /* Transforms in wide UV chars */ #define UNI_TO_NATIVE(ch) (ch) diff --git a/utfebcdic.h b/utfebcdic.h index c852946..003fb79 100644 --- a/utfebcdic.h +++ b/utfebcdic.h @@ -133,13 +133,11 @@ END_EXTERN_C /* EBCDIC-happy ways of converting native code to UTF-8 */ -/* Use these when ch is known to be < 256 */ -#define NATIVE_TO_LATIN1(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) PL_e2a[(U8)(ch)]) -#define LATIN1_TO_NATIVE(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) PL_a2e[(U8)(ch)]) +#define NATIVE_TO_LATIN1(ch) PL_e2a[(U8)(ch)] +#define LATIN1_TO_NATIVE(ch) PL_a2e[(U8)(ch)] -/* Use these on bytes */ -#define NATIVE_UTF8_TO_I8(b) (__ASSERT_(FITS_IN_8_BITS(b)) PL_e2utf[(U8)(b)]) -#define I8_TO_NATIVE_UTF8(b) (__ASSERT_(FITS_IN_8_BITS(b)) PL_utf2e[(U8)(b)]) +#define NATIVE_UTF8_TO_I8(ch) PL_e2utf[(U8)(ch)] +#define I8_TO_NATIVE_UTF8(ch) PL_utf2e[(U8)(ch)] /* Transforms in wide UV chars */ #define NATIVE_TO_UNI(ch) (FITS_IN_8_BITS(ch) ? NATIVE_TO_LATIN1(ch) : (ch)) -- 1.7.9.msysgit.0 ```

p5pRT commented 9 years ago

From @khwilliamson

On 09/12/2015 06:25 PM\, bulk88 (via RT) wrote:

# New Ticket Created by bulk88 # Please include the string: [perl #126045] # in the subject line of all future correspondence about this issue. # \<URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=126045 > This reverts commit e9b19ab7df3480e8f710ca6faad519f6fccdb081.

This fixes a Win32 Visual C 2003 DEBUGGING build failure in compiling regexec.obj.

cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regexec.obj -Fdmini\regexec.pdb ..\regexec.c regexec.c ..\regexec.c(2006) : fatal error C1001: INTERNAL COMPILER ERROR (compiler file 'msc1.cpp'\, line 2708) Please choose the Technical Support command on the Visual C++ Help menu\, or open the Technical Support help file for more information

No response from original author of the code at http://www.nntp.perl.org/group/perl.perl5.porters/2015/09/msg230842.html

Reverting the patch for the sake of a now-superseded compiler doesn't make sense. The patch detects array bounds errors on debugging builds on EBCDIC systems\, and on non-EBCDIC systems\, it detects code problems that would cause bounds errors when run on EBCDIC systems.

Of course\, we will try to support your compiler\, but support for current z/OS is no less worthy than support of an old\, superseded Win32 compiler. And\, the EBCDIC portion of the patch has zero effect on Windows platforms. There must be a better way to get your compiler to work than to revert this patch. If nothing else\, utf8.h could have #ifndef Win32 to work around the problematic code\, but I would think that there is a way to be more specific to limit things to just the problematic compilers.

Your final statement quoted above seems to indicate that because I (the original author) did not respond to an email you posted\, that you have no choice but to revert the whole patch. But I\, and most people\, aren't going to respond to non-actionable emails. You said\,

"I day or 2 ago on some a now deleted smoke-me branch of khws I got ...[the internal compiler error]."

There is nothing actionable in that. smoke-me branches come and go. This one was now gone. It seemed likely that this one was superseded by another one that had fixes in it. You gave no indication that you knew the failure was from the same set of changes that broke Merijn's compiler. I typically have several smokes of different things going on at the same time. I did not see anything I could do about this deleted code\, so didn't respond.

If we found out that the commit was stressing a bunch of different compilers\, then the non-EBCDIC portion might need to be reverted. But Merijn's compiler (for which we have a work-around) and yours are the only ones so far.

The bottom line is to submit a patch that gets your compiler to working\, but doesn't affect anyone else's.

This is a bug report for perl from bulk88@hotmail.com\, generated with the help of perlbug 1.40 running under perl 5.23.3.

----------------------------------------------------------------- [Please describe your issue here]

See attached patch.

before ---------------------------------------- C:\perl521\src\win32>dmake test CCTYPE=MSVC70 CFG=Debug if not exist ".\mini" mkdir ".\mini" if exist config.h del /f config.h copy config_H.vc config.h 1 file(s) copied. rem. > .\mini\.exists cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\toke.obj -Fdmini\toke.pdb ..\toke.c toke.c ..\toke.c(3118) : warning C4244: 'function' : conversion from 'IV' to 'const U8' \, possible loss of data ..\toke.c(3137) : warning C4244: 'function' : conversion from 'IV' to 'const U8' \, possible loss of data ..\toke.c(3660) : warning C4018: '>' : signed/unsigned mismatch cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regcomp.obj -Fdmini\regcomp.pdb ..\regcomp.c regcomp.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regexec.obj -Fdmini\regexec.pdb ..\regexec.c regexec.c ..\regexec.c(2006) : fatal error C1001: INTERNAL COMPILER ERROR (compiler file 'msc1.cpp'\, line 2708) Please choose the Technical Support command on the Visual C++ Help menu\, or open the Technical Support help file for more information

dmake: Error code 130\, while making 'mini\regexec.obj' C:\perl521\src\win32> ------------------------------------------- after ------------------------------------------- C:\perl521\src\win32>dmake test CCTYPE=MSVC70 CFG=Debug if not exist ".\mini" mkdir ".\mini" if exist config.h del /f config.h copy config_H.vc config.h 1 file(s) copied. rem. > .\mini\.exists cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\toke.obj -Fdmini\toke.pdb ..\toke.c toke.c ..\toke.c(3118) : warning C4244: 'function' : conversion from 'IV' to 'const U8' \, possible loss of data ..\toke.c(3137) : warning C4244: 'function' : conversion from 'IV' to 'const U8' \, possible loss of data ..\toke.c(3660) : warning C4018: '>' : signed/unsigned mismatch cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regcomp.obj -Fdmini\regcomp.pdb ..\regcomp.c regcomp.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regexec.obj -Fdmini\regexec.pdb ..\regexec.c regexec.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\op.obj -Fdmini\op.pdb ..\op.c op.c op.c(8397) : warning C4244: 'function' : conversion from 'cv_flags_t' to 'char'\, possible loss of data cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\sv.obj -Fdmini\sv.pdb ..\sv.c sv.c Caught SIGINT. Trying to quit ... dmake: C:\perl521\src\win32> -------------------------------------------

[Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=low --- Site configuration information for perl 5.23.3:

Configured by Owner at Sat Sep 12 17:01:50 2015.

Summary of my perl5 (revision 5 version 23 subversion 3) configuration: undef undef Platform: osname=MSWin32\, osvers=5.1\, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended\, useposix=true\, d_sigaction=undef useithreads=define\, usemultiplicity=define use64bitint=undef\, use64bitall=undef\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='cl'\, ccflags ='-nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32 -D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS'\, optimize='-O1 -MD -Zi -DNDEBUG -GL'\, cppflags='-DWIN32' ccversion='14.00.50727.42'\, gccversion=''\, gccosandvers='' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234\, doublekind=3 d_longlong=undef\, longlongsize=8\, d_longdbl=define\, longdblsize=8\, longdblkind=0 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='__int64'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='link'\, ldflags ='-nologo -nodefaultlib -debug -opt:ref\,icf -ltcg -libpath:"c:\perl\lib\CORE" -machine:x86 "/manifestdependency:type='Win32' name='Microsoft.Windows.Common-Controls' version='6.0.0.0' processorArchitecture='*' publicKeyToken='6595b64144ccf1df' language='*'"' libpth=\lib libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib libc=msvcrt.lib\, so=dll\, useshrplib=true\, libperl=perl523.lib gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs\, dlext=dll\, d_dlsymun=undef\, ccdlflags=' ' cccdlflags=' '\, lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref\,icf -ltcg -libpath:"c:\perl\lib\CORE" -machine:x86 "/manifestdependency:type='Win32' name='Microsoft.Windows.Common-Controls' version='6.0.0.0' processorArchitecture='*' publicKeyToken='6595b64144ccf1df' language='*'"'

--- @INC for perl 5.23.3: C:\perl521\src\lib C:\perl521\src\cpan\AutoLoader\lib C:\perl521\src\dist\Carp\lib C:\perl521\src\dist\PathTools C:\perl521\src\dist\PathTools\lib C:\perl521\src\cpan\ExtUtils-Command\lib C:\perl521\src\cpan\ExtUtils-Install\lib C:\perl521\src\cpan\ExtUtils-MakeMaker\lib C:\perl521\src\cpan\ExtUtils-Manifest\lib C:\perl521\src\cpan\File-Path\lib C:\perl521\src\ext\re C:\perl521\src\dist\Term-ReadLine\lib C:\perl521\src\dist\Exporter\lib C:\perl521\src\ext\File-Find\lib C:\perl521\src\cpan\Text-Tabs\lib C:\perl521\src\dist\constant\lib C:\perl521\src\cpan\Text-ParseWords\lib C:\perl521\src\dist\ExtUtils-ParseXS\lib C:\perl521\src\cpan\Getopt-Long\lib C:\perl521\src\cpan\parent\lib C:\perl521\src\cpan\ExtUtils-Constant\lib .

--- Environment for perl 5.23.3: HOME (unset) LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=C:\perl521\bin;C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\IDE;C:\Program Files\Microsoft Visual Studio .NET 2003\VC7\BIN;C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools;C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools\bin\prerelease;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\system32\wbem; PERL_BADLANG (unset) SHELL (unset)

p5pRT commented 9 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 9 years ago

From @bulk88

On Thu Sep 17 20:36:24 2015\, public@khwilliamson.com wrote:

No response from original author of the code at http://www.nntp.perl.org/group/perl.perl5.porters/2015/09/msg230842.html

Reverting the patch for the sake of a now-superseded compiler doesn't make sense.

P5P supports VC 6 to 2013. Are you announcing that Perl 5.24 has discontinued supporting Visual C?

The patch detects array bounds errors on debugging builds on EBCDIC systems\, and on non-EBCDIC systems\, it detects code problems that would cause bounds errors when run on EBCDIC systems.

Of course\, we will try to support your compiler\, but support for current z/OS is no less worthy than support of an old\, superseded Win32 compiler. And\, the EBCDIC portion of the patch has zero effect on Windows platforms. There must be a better way to get your compiler to work than to revert this patch. If nothing else\, utf8.h could have #ifndef Win32 to work around the problematic code\, but I would think that there is a way to be more specific to limit things to just the problematic compilers.

Your final statement quoted above seems to indicate that because I (the original author) did not respond to an email you posted\, that you have no choice but to revert the whole patch. But I\, and most people\, aren't going to respond to non-actionable emails. You said\,

"I day or 2 ago on some a now deleted smoke-me branch of khws I got ...[the internal compiler error]."

There is nothing actionable in that. smoke-me branches come and go. This one was now gone. It seemed likely that this one was superseded by another one that had fixes in it. You gave no indication that you knew the failure was from the same set of changes that broke Merijn's compiler. I typically have several smokes of different things going on at the same time. I did not see anything I could do about this deleted code\, so didn't respond.

If you didn't change anything to address my complaint\, and you pushed the code to blead\, why would it suddenly stop failing? The only fix that was ever determined in that thread was to change the CC's buffer size for HP CC ( http://www.nntp.perl.org/group/perl.perl5.porters/2015/09/msg230941.html ) and nothing was committed to blead since Tux said he needs to test it further. You could have asked/posted "can you still reproduce this problem?" or "file a bug ticket so this isn't forgotten". Instead you never responded at all to me.

If we found out that the commit was stressing a bunch of different compilers\, then the non-EBCDIC portion might need to be reverted. But Merijn's compiler (for which we have a work-around) and yours are the only ones so far.

You broke 2 different CCs and didnt know about it until the smoke tests/user reports came in. Perhaps the design of the code/that patch should be rethought instead of figuring out how to fix breakage after the fact and assuming the patch is a golden cow. Using rare uncommon options to fix something says the code is difficult to maintain\, and the code should change to meet the CC/interpreter/parser whatever and not "break" on every "upgrade". For that patch an "upgrade" is some new CC or port in the future.

The bottom line is to submit a patch that gets your compiler to working\, but doesn't affect anyone else's.

Why dont you submit a patch that gets everyone elses compiler working? You are the author of the LOC and the author of the API (FBC_BOUND) in question. If you didn't have a commit bit\, your patch would be reverted within a few days if you failed to respond/provide a fix\, am I not right?

I did research fixing it myself\, but I stopped since you created an macro API (macro FBC_BOUND) that uses implicitly assumes there are certain C autos in scope\, which is not allowed by perl's design style policy (there is grandfathered use AFAIK). At that point I stopped thinking of writing a patch since I cant change FBC_BOUND and started bisecting.

FBC_BOUND creation commit http://perl5.git.perl.org/perl.git/commitdiff/63ac0dadb1aafcf0c171d3c1422c1923b611b2fc

Blead at http://perl5.git.perl.org/perl.git/b58e9a87f1dde6553a833962d54161262ba90be5 still has the build failure so it has not been fixed accidentally/unintentionally and this revert patch is still valid.

C:\perl521\src\win32>dmake mini/regexec.obj CCTYPE=MSVC70 CFG=Debug if not exist ".\mini" mkdir ".\mini" if exist config.h del /f config.h copy config_H.vc config.h 1 file(s) copied. rem. > .\mini\.exists cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regexec.obj -Fdmini\regexec.pdb ..\regexec.c regexec.c ..\regexec.c(2006) : fatal error C1001: INTERNAL COMPILER ERROR (compiler file 'msc1.cpp'\, line 2708) Please choose the Technical Support command on the Visual C++ Help menu\, or open the Technical Support help file for more information

dmake: Error code 130\, while making 'mini\regexec.obj'

C:\perl521\src\win32>

The technical problem is "FBC_BOUND(isWORDCHAR_LC\, isWORDCHAR_LC_uvchr\, isWORDCHAR_LC_utf8);" is expanding beyond a fixed length buffer in the VC/HP C CPP code. HP C has a buffer change cmd line option\, VC does not. For VC the fixed length buffer is probably some power of 2 between 4KB and 64KB including those 2.

As davem suggested in http://www.nntp.perl.org/group/perl.perl5.porters/2015/09/msg230968.html you need to turn stuff into functions. You are the creator of every token in the string "FBC_BOUND(isWORDCHAR_LC\, isWORDCHAR_LC_uvchr\, isWORDCHAR_LC_utf8);"

http://perl5.git.perl.org/perl.git/commitdiff/63ac0dadb1aafcf0c171d3c1422c1923b611b2fc http://perl5.git.perl.org/perl.git/commitdiff/ca5b5914906cf989a5d5a21f2af8842ab69eb369

Nobody else is going to\, or will be competent at refactoring that API except you.

Some people have said on P5P they use DEBUGGING perl for their business code. I dont care about optimizing DEBUGGING perl and never tried\, but maybe the speed of DEBUGGING perl should be paid attention to since it affects

1. people paranoid enough to use OpenBSD and/or 100% asserts in all binaries on their system 2. smoke infrastructure/waiting for results

FBC_BOUND expands to pagefuls and pagefuls of code. A CC has to optimize all that code when creating regexec.o slowing down regexec.o compile time (its #3 in perl core on GCC compile time). Also there is a high risk that the same assert check runs multiple times after -O2 optimization due to macros in multieval macros being exponentially expanded. I suspect FBC_BOUND looks more like it should be a function call that calls func pointers from objects since it implements an algorithm that parts are plugged into.

Useless details ahead about Visual C since it is closed source.

A callstack of the assert in Visual C doesn't show enough to be very useful.

c1.dll!Trap_main_compile() + 0x14e39
c1.dll!CallMain() + 0x139
c1.dll!Trap_CallMain() + 0x37
c1.dll!_InvokeCompilerPass@12() + 0x21
cl.exe!execute() + 0x1c1
cl.exe!dopass() + 0x2be
cl.exe!passes() + 0x38
cl.exe!post_compile() + 0x269
cl.exe!compile() + 0xad
cl.exe!_main() + 0x4c0
cl.exe!_mainCRTStartup() + 0x143
kernel32.dll!_BaseProcessStart@4() + 0x23

A SEGV exception in the CC.

c1.dll!tl_getid() - 0xaac7
c1.dll!can_expand() - 0xb43
c1.dll!GetTokenFromCurrentChar() + 0x42b5 AKA ?LexerResolveIdentifier@@YIPBVToken@@PBV1@_NPA_N@Z: e87c3a21()

First-chance exception at 0x1061af14 (c1.dll) in cl.exe: 0xC0000005: Access violation writing location 0x008c4000. First-chance exception at 0x1061b034 (c1.dll) in cl.exe: 0xC0000005: Access violation writing location 0x008c8000. First-chance exception at 0x1061ffbe (c1.dll) in cl.exe: 0xC0000005: Access violation writing location 0x00dd4000. First-chance exception at 0x1061ffbe (c1.dll) in cl.exe: 0xC0000005: Access violation writing location 0x00dd8000. First-chance exception at 0x1061b16e (c1.dll) in cl.exe: 0xC0000005: Access violation reading location 0x008bb664.

The first couple SEGVs\, note all of them are at exactly a page boundary\, those pages are allocated by a SEGV handler inside Visual C and execution resumes normally. 0x8c0000 and 0xdd0000 are 5124 KB blocks with only a couple 4KB chunks allocated at the beginning of the 5124KB reserved region/mem obj region. 0x8c0000 and 0xdd0000 memory regions are marked private/directly mmaped/directly VirtualAlloced by the OS\, they are not part of malloc. The last SEGV is not at an page boundary and probably is causing the assert fail.

Mem obj 0x8b0000 is a region from 0x8b0000 to 0x8c0000\, 64KB big. 0x8b0000 to 0x8b5000 (20KB) is allocated and part of malloc. 0x8b5000 to 0x8c0000 is reserved for malloc to expand into\, and it is owned by the malloc code. Faulting address pointer 0x008bb664 is a overflow by 0x008bb664 - 0x8b5000 = 0x6664 bytes or ~26 KB.

The GCC expansion of "FBC_BOUND(isWORDCHAR_LC\, isWORDCHAR_LC_uvchr\, isWORDCHAR_LC_utf8);" winds up being 49476 bytes long for me. The VC expansion will be different but IDK if larger or smaller.

Telling malloc to dump itself.

0:001> !heap -h 0x8b0000 Index Address Name Debugging options enabled 6: 008b0000 Segment at 008b0000 to 008c0000 (00005000 bytes committed) Flags: 00001002 ForceFlags: 00000000 Granularity: 8 bytes Segment Reserve: 00100000 Segment Commit: 00002000 DeCommit Block Thres: 00000200 DeCommit Total Thres: 00002000 Total Free Size: 000001cb Max. Allocation Size: 7ffdefff Lock Variable at: 008b0608 Next TagIndex: 0000 Maximum TagIndex: 0000 Tag Entries: 00000000 PsuedoTag Entries: 00000000 Virtual Alloc List: 008b0050 UCR FreeList: 008b0598 FreeList Usage: 00000080 00000000 00000000 00000000 FreeList[ 00 ] at 008b0178: 008b24c0 . 008b24c0 (1 block ) FreeList[ 07 ] at 008b01b0: 008b4fd0 . 008b4fd0 (1 block ) Heap entries for Segment00 in Heap 008b0000 008b0640: 00640 . 00040 [01] - busy (40) 008b0680: 00040 . 01808 [01] - busy (1800) 008b1e88: 01808 . 00090 [01] - busy (88) 008b1f18: 00090 . 00488 [01] - busy (480) 008b23a0: 00488 . 00088 [01] - busy (80) 008b2428: 00088 . 00090 [01] - busy (88) 008b24b8: 00090 . 00e20 [00] 008b32d8: 00e20 . 00228 [01] - busy (220) 008b3500: 00228 . 000e0 [01] - busy (d4) 008b35e0: 000e0 . 00050 [01] - busy (41) 008b3630: 00050 . 00040 [01] - busy (34) 008b3670: 00040 . 00048 [01] - busy (39) 008b36b8: 00048 . 00048 [01] - busy (3f) 008b3700: 00048 . 00020 [01] - busy (13) 008b3720: 00020 . 00040 [01] - busy (31) 008b3760: 00040 . 00028 [01] - busy (1d) 008b3788: 00028 . 00030 [01] - busy (24) 008b37b8: 00030 . 00058 [01] - busy (49) 008b3810: 00058 . 00020 [01] - busy (14) 008b3830: 00020 . 00038 [01] - busy (30) 008b3868: 00038 . 00058 [01] - busy (4c) 008b38c0: 00058 . 00028 [01] - busy (1b) 008b38e8: 00028 . 00018 [01] - busy (e) 008b3900: 00018 . 00038 [01] - busy (29) 008b3938: 00038 . 00018 [01] - busy (d) 008b3950: 00018 . 00030 [01] - busy (27) 008b3980: 00030 . 00180 [01] - busy (173) 008b3b00: 00180 . 00168 [01] - busy (15b) 008b3c68: 00168 . 00028 [01] - busy (1e) 008b3c90: 00028 . 00048 [01] - busy (3f) 008b3cd8: 00048 . 00028 [01] - busy (1b) 008b3d00: 00028 . 00020 [01] - busy (17) 008b3d20: 00020 . 00040 [01] - busy (36) 008b3d60: 00040 . 00018 [01] - busy (e) 008b3d78: 00018 . 004f8 [01] - busy (4ef) 008b4270: 004f8 . 00048 [01] - busy (39) 008b42b8: 00048 . 00030 [01] - busy (23) 008b42e8: 00030 . 00020 [01] - busy (17) 008b4308: 00020 . 00028 [01] - busy (1b) 008b4330: 00028 . 00050 [01] - busy (44) 008b4380: 00050 . 00020 [01] - busy (12) 008b43a0: 00020 . 00020 [01] - busy (18) 008b43c0: 00020 . 00028 [01] - busy (1e) 008b43e8: 00028 . 00018 [01] - busy (c) 008b4400: 00018 . 00020 [01] - busy (14) 008b4420: 00020 . 00038 [01] - busy (29) 008b4458: 00038 . 00018 [01] - busy (f) 008b4470: 00018 . 00020 [01] - busy (16) 008b4490: 00020 . 00020 [01] - busy (15) 008b44b0: 00020 . 00018 [01] - busy (a) 008b44c8: 00018 . 00020 [01] - busy (14) 008b44e8: 00020 . 00028 [01] - busy (1b) 008b4510: 00028 . 00018 [01] - busy (f) 008b4528: 00018 . 00038 [01] - busy (2c) 008b4560: 00038 . 00048 [01] - busy (40) 008b45a8: 00048 . 00058 [01] - busy (50) 008b4600: 00058 . 00050 [01] - busy (48) 008b4650: 00050 . 00058 [01] - busy (4c) 008b46a8: 00058 . 00020 [01] - busy (12) 008b46c8: 00020 . 00010 [01] - busy (6) 008b46d8: 00010 . 00060 [01] - busy (53) 008b4738: 00060 . 00088 [01] - busy (80) 008b47c0: 00088 . 00808 [01] - busy (800) 008b4fc8: 00808 . 00038 [10] 008b5000: 0000b000 - uncommitted bytes.

number in parens is malloc allocation size in hex. If I go plainly by size of the last entry\, 0x800 or 2KB\, that is the buffer size of the Visual C C preprocessor. The real answer might be more complicated with 0x800 being a truncated 0x10800 or 0x1800. Notice the 0x1800 allocation also. It was filled with alot 0x01\, 0x04\, and mostly 0x00 bytes\, nothing else. The last in malloc heap big 0x800 allocation is a bunch of FILE * from libc. 0x77c5____ are data globals inside msvcrt.dll\, including FILE * fd 0\,1\,2. They might be uninitialized memory from the past.

008b47c0 01 01 11 00 f0 01 08 00 80 fc c5 77 a0 fc c5 77 c0 fc ...........w...w.. 008b47d2 c5 77 e0 fc c5 77 00 fd c5 77 20 fd c5 77 40 fd c5 77 .w...w...w ..w@..w 008b47e4 60 fd c5 77 80 fd c5 77 a0 fd c5 77 c0 fd c5 77 e0 fd `..w...w...w...w.. 008b47f6 c5 77 00 fe c5 77 20 fe c5 77 40 fe c5 77 60 fe c5 77 .w...w ..w@..w`..w 008b4808 80 fe c5 77 a0 fe c5 77 c0 fe c5 77 e0 fe c5 77 00 00 ...w...w...w...w.. 008b481a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b482c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b483e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b4850 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b4862 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b4874 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b4886 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b4898 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b48aa 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b48bc 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b48ce 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b48e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b48f2 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b4904 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b4916 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .................. 008b4928 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ..................

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 8 years ago

From @tonycoz

On Thu Sep 24 04:11:58 2015\, bulk88 wrote:

I did research fixing it myself\, but I stopped since you created an macro API (macro FBC_BOUND) that uses implicitly assumes there are certain C autos in scope\, which is not allowed by perl's design style policy (there is grandfathered use AFAIK). At that point I stopped thinking of writing a patch since I cant change FBC_BOUND and started bisecting.

As davem suggested in http://www.nntp.perl.org/group/perl.perl5.porters/2015/09/msg230968.html you need to turn stuff into functions. You are the creator of every token in the string "FBC_BOUND(isWORDCHAR_LC\, isWORDCHAR_LC_uvchr\, isWORDCHAR_LC_utf8);"

... Nobody else is going to\, or will be competent at refactoring that API except you.

I (and I expect khw) don't have the compiler to test with\, and it's not being used for smoking.

Only you can test such changes.

I did research fixing it myself\, but I stopped since you created an macro API (macro FBC_BOUND) that uses implicitly assumes there are certain C autos in scope\, which is not allowed by perl's design style policy (there is grandfathered use AFAIK).

While I think it's bad in general\, both the parser and the regexp engine make use of it\, and for good reason I think.

Could you please test the branch tonyc/vc2003-macro-issues to see if it crashes VC2003?

Tony

p5pRT commented 8 years ago

From @tonycoz

0001-WIP-test-fix-for-126045-needs-work-needs-testing.patch

```diff From e708ca371e3916d4d43873b94590507b76536a3b Mon Sep 17 00:00:00 2001 From: Tony Cook Date: Tue, 3 Nov 2015 15:27:31 +1100 Subject: WIP test fix for 126045, needs work, needs testing --- regexec.c | 48 +++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 43 insertions(+), 5 deletions(-) diff --git a/regexec.c b/regexec.c index 85c31a6..c76190f 100644 --- a/regexec.c +++ b/regexec.c @@ -263,6 +263,44 @@ static void S_setup_eval_state(pTHX_ regmatch_info *const reginfo); static void S_cleanup_regmatch_info_aux(pTHX_ void *arg); static regmatch_state * S_push_slab(pTHX); +/* this needs to check the correct version number for MSVC 2003 */ +#ifdef _MSC_VER + +PERL_STATIC_INLINE bool +S_isWORDCHAR_uni(pTHX_ UV c) { + /* macro already does cBOOL() */ + return isWORDCHAR_uni(c); +} + +PERL_STATIC_INLINE bool +S_isWORDCHAR_LC_uvchr(pTHX_ UV c) { + return isWORDCHAR_LC_uvchr(c); +} + +PERL_STATIC_INLINE bool +S_isWORDCHAR_utf8(pTHX_ const U8 *p) { + return isWORDCHAR_utf8(p); +} + +PERL_STATIC_INLINE bool +S_isWORDCHAR_LC_utf8(pTHX_ const U8 *p) { + return isWORDCHAR_LC_utf8(p); +} + +#define isWORDCHAR_uni_x(c) (S_isWORDCHAR_uni(aTHX_ (c))) +#define isWORDCHAR_LC_uvchr_x(c) (S_isWORDCHAR_LC_uvchr(aTHX_ (c))) +#define isWORDCHAR_utf8_x(p) (S_isWORDCHAR_utf8(aTHX_ (p))) +#define isWORDCHAR_LC_utf8_x(p) (S_isWORDCHAR_LC_utf8(aTHX_ (p))) + +#else + +#define isWORDCHAR_uni_x isWORDCHAR_uni +#define isWORDCHAR_LC_uvchr_x isWORDCHAR_LC_uvchr +#define isWORDCHAR_utf8_x isWORDCHAR_utf8 +#define isWORDCHAR_LC_utf8_x isWORDCHAR_LC_utf8 + +#endif + #define REGCP_PAREN_ELEMS 3 #define REGCP_OTHER_ELEMS 3 #define REGCP_FRAME_ELEMS 1 @@ -2003,7 +2041,7 @@ S_find_byclass(pTHX_ regexp * prog, const regnode *c, char *s, goto do_boundu; } - FBC_BOUND(isWORDCHAR_LC, isWORDCHAR_LC_uvchr, isWORDCHAR_LC_utf8); + FBC_BOUND(isWORDCHAR_LC, isWORDCHAR_LC_uvchr_x, isWORDCHAR_LC_utf8_x); break; case NBOUNDL: @@ -2016,14 +2054,14 @@ S_find_byclass(pTHX_ regexp * prog, const regnode *c, char *s, goto do_nboundu; } - FBC_NBOUND(isWORDCHAR_LC, isWORDCHAR_LC_uvchr, isWORDCHAR_LC_utf8); + FBC_NBOUND(isWORDCHAR_LC, isWORDCHAR_LC_uvchr_x, isWORDCHAR_LC_utf8_x); break; case BOUND: /* regcomp.c makes sure that this only has the traditional \b meaning */ assert(FLAGS(c) == TRADITIONAL_BOUND); - FBC_BOUND(isWORDCHAR, isWORDCHAR_uni, isWORDCHAR_utf8); + FBC_BOUND(isWORDCHAR, isWORDCHAR_uni_x, isWORDCHAR_utf8_x); break; case BOUNDA: /* regcomp.c makes sure that this only has the traditional \b @@ -2037,7 +2075,7 @@ S_find_byclass(pTHX_ regexp * prog, const regnode *c, char *s, meaning */ assert(FLAGS(c) == TRADITIONAL_BOUND); - FBC_NBOUND(isWORDCHAR, isWORDCHAR_uni, isWORDCHAR_utf8); + FBC_NBOUND(isWORDCHAR, isWORDCHAR_uni_x, isWORDCHAR_utf8_x); break; case NBOUNDA: /* regcomp.c makes sure that this only has the traditional \b @@ -2049,7 +2087,7 @@ S_find_byclass(pTHX_ regexp * prog, const regnode *c, char *s, case NBOUNDU: if ((bound_type) FLAGS(c) == TRADITIONAL_BOUND) { - FBC_NBOUND(isWORDCHAR_L1, isWORDCHAR_uni, isWORDCHAR_utf8); + FBC_NBOUND(isWORDCHAR_L1, isWORDCHAR_uni_x, isWORDCHAR_utf8_x); break; } -- 2.1.4 ```

p5pRT commented 8 years ago

From @bulk88

On Mon Nov 02 21:02:44 2015\, tonyc wrote:

While I think it's bad in general\, both the parser and the regexp engine make use of it\, and for good reason I think.

Could you please test the branch tonyc/vc2003-macro-issues to see if it crashes VC2003?

Tony

Your branch changes the fatal error from VC 2003 but doesnt fix it.

before

C:\perl521\src\win32>dmake test if not exist ".\mini" mkdir ".\mini" if exist config.h del /f config.h copy config_H.vc config.h 1 file(s) copied. rem. > .\mini\.exists cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\toke.obj -Fdmini\toke.pdb ..\toke.c toke.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regcomp.obj -Fdmini\regcomp.pdb ..\regcomp.c regcomp.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regexec.obj -Fdmini\regexec.pdb ..\regexec.c regexec.c ..\regexec.c(2006) : fatal error C1001: INTERNAL COMPILER ERROR (compiler file 'msc1.cpp'\, line 2708) Please choose the Technical Support command on the Visual C++ Help menu\, or open the Technical Support help file for more information

dmake: Error code 130\, while making 'mini\regexec.obj'

C:\perl521\src\win32>

after your patch "* WIP test fix for 126045\, needs work\, needs testing"

C:\perl521\src\win32>dmake test if not exist ".\mini" mkdir ".\mini" if exist config.h del /f config.h copy config_H.vc config.h 1 file(s) copied. rem. > .\mini\.exists cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\toke.obj -Fdmini\toke.pdb ..\toke.c toke.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regcomp.obj -Fdmini\regcomp.pdb ..\regcomp.c regcomp.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regexec.obj -Fdmini\regexec.pdb ..\regexec.c regexec.c ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5671) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5671) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5671) : error C2026: string too big\, trailing characters truncated dmake: Error code 130\, while making 'mini\regexec.obj'

C:\perl521\src\win32>

The description for C2026 says the buffer is 2KB. https://msdn.microsoft.com/en-us/library/dddywwsc%28v=vs.71%29.aspx

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 8 years ago

From @bulk88

On Sat Nov 28 00:40:08 2015\, bulk88 wrote:

On Mon Nov 02 21:02:44 2015\, tonyc wrote:

While I think it's bad in general\, both the parser and the regexp engine make use of it\, and for good reason I think.

Could you please test the branch tonyc/vc2003-macro-issues to see if it crashes VC2003?

Tony

Your branch changes the fatal error from VC 2003 but doesnt fix it.

before ------------------------------------------------------ C:\perl521\src\win32>dmake test if not exist ".\mini" mkdir ".\mini" if exist config.h del /f config.h copy config_H.vc config.h 1 file(s) copied. rem. > .\mini\.exists cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\toke.obj -Fdmini\toke.pdb ..\toke.c toke.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regcomp.obj -Fdmini\regcomp.pdb ..\regcomp.c regcomp.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regexec.obj -Fdmini\regexec.pdb ..\regexec.c regexec.c ..\regexec.c(2006) : fatal error C1001: INTERNAL COMPILER ERROR (compiler file 'msc1.cpp'\, line 2708) Please choose the Technical Support command on the Visual C++ Help menu\, or open the Technical Support help file for more information

dmake: Error code 130\, while making 'mini\regexec.obj'

C:\perl521\src\win32> ------------------------------------------------------

after your patch "* WIP test fix for 126045\, needs work\, needs testing" ------------------------------------------------------ C:\perl521\src\win32>dmake test if not exist ".\mini" mkdir ".\mini" if exist config.h del /f config.h copy config_H.vc config.h 1 file(s) copied. rem. > .\mini\.exists cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\toke.obj -Fdmini\toke.pdb ..\toke.c toke.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regcomp.obj -Fdmini\regcomp.pdb ..\regcomp.c regcomp.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -Od -MD -Zi -DDEBUGGING -DPERL_EXTERNAL_GLOB - DPERL_IS_MINIPERL -Fomini\regexec.obj -Fdmini\regexec.pdb ..\regexec.c regexec.c ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5592) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5671) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5671) : error C2026: string too big\, trailing characters truncated ..\regexec.c(5671) : error C2026: string too big\, trailing characters truncated dmake: Error code 130\, while making 'mini\regexec.obj'

C:\perl521\src\win32> ------------------------------------------------------

The description for C2026 says the buffer is 2KB. https://msdn.microsoft.com/en-us/library/dddywwsc%28v=vs.71%29.aspx

Bump

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 8 years ago

From @tonycoz

On Sat Nov 28 00:40:08 2015\, bulk88 wrote:

On Mon Nov 02 21:02:44 2015\, tonyc wrote:

While I think it's bad in general\, both the parser and the regexp engine make use of it\, and for good reason I think.

Could you please test the branch tonyc/vc2003-macro-issues to see if it crashes VC2003?

Tony

Your branch changes the fatal error from VC 2003 but doesnt fix it.

How about the attached?

This follows the approach khw suggested in IRC\, assuming I understand what he meant.

The value I'm checking _MSC_VER against came from http://varlab.blogspot.com.au/2012/11/if-you-develop-applications-for-windows.html so it might need adjustment if that page is incorrect.

Tony

p5pRT commented 8 years ago

From @tonycoz

0001-perl-126045-part-revert-e9b19ab7-for-vc2003-only.patch

```diff From 31c4a31a656328fc3509a6a2abfddf13b345e305 Mon Sep 17 00:00:00 2001 From: Tony Cook Date: Mon, 4 Jan 2016 15:50:59 +1100 Subject: [PATCH] [perl #126045] part revert e9b19ab7 for vc2003 only This avoids an internal compiler error on VC 2003. --- utf8.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/utf8.h b/utf8.h index 1ed8fd8..5af7dc3 100644 --- a/utf8.h +++ b/utf8.h @@ -136,8 +136,13 @@ END_EXTERN_C /* Native character to/from iso-8859-1. Are the identity functions on ASCII * platforms */ +#if _MSC_VER == 1310 +#define NATIVE_TO_LATIN1(ch) (ch) +#define LATIN1_TO_NATIVE(ch) (ch) +#else #define NATIVE_TO_LATIN1(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) ((U8) (ch))) #define LATIN1_TO_NATIVE(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) ((U8) (ch))) +#endif /* I8 is an intermediate version of UTF-8 used only in UTF-EBCDIC. We thus * consider it to be identical to UTF-8 on ASCII platforms. Strictly speaking @@ -145,8 +150,13 @@ END_EXTERN_C * because they are 8-bit encodings that serve the same purpose in Perl, and * rarely do we need to distinguish them. The term "NATIVE_UTF8" applies to * whichever one is applicable on the current platform */ +#if _MSC_VER == 1310 +#define NATIVE_UTF8_TO_I8(ch) (ch) +#define I8_TO_NATIVE_UTF8(ch) (ch) +#else #define NATIVE_UTF8_TO_I8(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) ((U8) (ch))) #define I8_TO_NATIVE_UTF8(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) ((U8) (ch))) +#endif /* Transforms in wide UV chars */ #define UNI_TO_NATIVE(ch) ((UV) (ch)) -- 1.9.5.msysgit.0 ```

p5pRT commented 8 years ago

From @bulk88

On Sun Jan 03 20:54:25 2016\, tonyc wrote:

On Sat Nov 28 00:40:08 2015\, bulk88 wrote:

On Mon Nov 02 21:02:44 2015\, tonyc wrote:

While I think it's bad in general\, both the parser and the regexp engine make use of it\, and for good reason I think.

Could you please test the branch tonyc/vc2003-macro-issues to see if it crashes VC2003?

Tony

Your branch changes the fatal error from VC 2003 but doesnt fix it.

How about the attached?

This follows the approach khw suggested in IRC\, assuming I understand what he meant.

The value I'm checking _MSC_VER against came from http://varlab.blogspot.com.au/2012/11/if-you-develop-applications-for- windows.html so it might need adjustment if that page is incorrect.

Tony

the latest patch "part revert e9b19ab7 for vc2003 only" fixes the problem.

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 8 years ago

From @bulk88

On Fri Jan 15 09:28:58 2016\, bulk88 wrote:

On Sun Jan 03 20:54:25 2016\, tonyc wrote:

On Sat Nov 28 00:40:08 2015\, bulk88 wrote:

On Mon Nov 02 21:02:44 2015\, tonyc wrote:

While I think it's bad in general\, both the parser and the regexp engine make use of it\, and for good reason I think.

Could you please test the branch tonyc/vc2003-macro-issues to see if it crashes VC2003?

Tony

Your branch changes the fatal error from VC 2003 but doesnt fix it.

How about the attached?

This follows the approach khw suggested in IRC\, assuming I understand what he meant.

The value I'm checking _MSC_VER against came from http://varlab.blogspot.com.au/2012/11/if-you-develop-applications-for- windows.html so it might need adjustment if that page is incorrect.

Tony

the latest patch "part revert e9b19ab7 for vc2003 only" fixes the problem.

Egh\, I tested VC6\, the CC crashed like VC 2003 does\, you need to add VC 6 and 2002 (dont have it) to the no assert logic . I tested VC 2005 and it built fine (no crash\, I guess VC 2005 is the first VC to increase its preprocessor buffer size).

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 8 years ago

From @tonycoz

On Fri Jan 15 09:38:03 2016\, bulk88 wrote:

On Fri Jan 15 09:28:58 2016\, bulk88 wrote:

the latest patch "part revert e9b19ab7 for vc2003 only" fixes the problem.

Egh\, I tested VC6\, the CC crashed like VC 2003 does\, you need to add VC 6 and 2002 (dont have it) to the no assert logic . I tested VC 2005 and it built fine (no crash\, I guess VC 2005 is the first VC to increase its preprocessor buffer size).

Here's an updated patch.

Tony

p5pRT commented 8 years ago

From @tonycoz

0001-perl-126045-part-revert-e9b19ab7-for-vc2003-and-earl.patch

```diff From 228f19ed730b470b3ca2afbbcbed00da8f646466 Mon Sep 17 00:00:00 2001 From: Tony Cook Date: Thu, 28 Jan 2016 15:08:01 +1100 Subject: [perl #126045] part revert e9b19ab7 for vc2003 and earlier This avoids an internal compiler error on VC 2003 and earlier --- utf8.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/utf8.h b/utf8.h index 1ed8fd8..e523cc3 100644 --- a/utf8.h +++ b/utf8.h @@ -134,10 +134,20 @@ EXTCONST unsigned char PL_utf8skip[]; END_EXTERN_C +#if _MSC_VER < 1400 +/* older MSVC versions have a smallish macro buffer */ +#define PERL_SMALL_MACRO_BUFFER +#endif + /* Native character to/from iso-8859-1. Are the identity functions on ASCII * platforms */ +#ifdef PERL_SMALL_MACRO_BUFFER +#define NATIVE_TO_LATIN1(ch) (ch) +#define LATIN1_TO_NATIVE(ch) (ch) +#else #define NATIVE_TO_LATIN1(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) ((U8) (ch))) #define LATIN1_TO_NATIVE(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) ((U8) (ch))) +#endif /* I8 is an intermediate version of UTF-8 used only in UTF-EBCDIC. We thus * consider it to be identical to UTF-8 on ASCII platforms. Strictly speaking @@ -145,8 +155,13 @@ END_EXTERN_C * because they are 8-bit encodings that serve the same purpose in Perl, and * rarely do we need to distinguish them. The term "NATIVE_UTF8" applies to * whichever one is applicable on the current platform */ +#ifdef PERL_SMALL_MACRO_BUFFER +#define NATIVE_UTF8_TO_I8(ch) (ch) +#define I8_TO_NATIVE_UTF8(ch) (ch) +#else #define NATIVE_UTF8_TO_I8(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) ((U8) (ch))) #define I8_TO_NATIVE_UTF8(ch) (__ASSERT_(FITS_IN_8_BITS(ch)) ((U8) (ch))) +#endif /* Transforms in wide UV chars */ #define UNI_TO_NATIVE(ch) ((UV) (ch)) -- 1.9.5.msysgit.0 ```

p5pRT commented 8 years ago

From @bulk88

On Wed Jan 27 20:09:11 2016\, tonyc wrote:

On Fri Jan 15 09:38:03 2016\, bulk88 wrote:

On Fri Jan 15 09:28:58 2016\, bulk88 wrote:

the latest patch "part revert e9b19ab7 for vc2003 only" fixes the problem.

Egh\, I tested VC6\, the CC crashed like VC 2003 does\, you need to add VC 6 and 2002 (dont have it) to the no assert logic . I tested VC 2005 and it built fine (no crash\, I guess VC 2005 is the first VC to increase its preprocessor buffer size).

Here's an updated patch.

Tony

new patch works on VC 6 and VC 2003 with DEBUGGING. no CC SEGV or errors.

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 8 years ago

From @tonycoz

On Thu Jan 28 03:01:28 2016\, bulk88 wrote:

On Wed Jan 27 20:09:11 2016\, tonyc wrote:

On Fri Jan 15 09:38:03 2016\, bulk88 wrote:

On Fri Jan 15 09:28:58 2016\, bulk88 wrote:

the latest patch "part revert e9b19ab7 for vc2003 only" fixes the problem.

Egh\, I tested VC6\, the CC crashed like VC 2003 does\, you need to add VC 6 and 2002 (dont have it) to the no assert logic . I tested VC 2005 and it built fine (no crash\, I guess VC 2005 is the first VC to increase its preprocessor buffer size).

Here's an updated patch.

new patch works on VC 6 and VC 2003 with DEBUGGING. no CC SEGV or errors.

Thanks for testing\, applied as 6f6d1bab3343e4cff9dab0934b654ade4a85bd59.

Tony

p5pRT commented 8 years ago

@tonycoz - Status changed from 'open' to 'pending release'

p5pRT commented 8 years ago

From @khwilliamson

Thank you for submitting this report. You have helped make Perl better.
With the release of Perl 5.24.0 on May 9\, 2016\, this and 149 other issues have been resolved.

Perl 5.24.0 may be downloaded via https://metacpan.org/release/RJBS/perl-5.24.0

p5pRT commented 8 years ago

@khwilliamson - Status changed from 'pending release' to 'resolved'

Perl / perl5