Closed p5pRT closed 7 years ago
This ticket is for centralizing discussions about how to add support for building with VS2015.
There have been a couple of p5p threads already; it is better to keep it all in one place:
http://www.nntp.perl.org/group/perl.perl5.porters/2015/05/msg227702.html http://www.nntp.perl.org/group/perl.perl5.porters/2015/07/msg229524.html
Attached is my first attempt\, originally posted in the latter thread above (saved here since http://www.nntp.perl.org doesn't seem to save attachments).
Also attached is my second attempt: This does surprisingly well with the quick-and-dirty approach of simply casting a FILE* to the appropriate new CRT struct (__crt_stdio_stream_data) and accessing the members of that. It might even suffice\, rather than going to the hassle of writing accessors and probably needing to separate getters from setters?
However\, the build currently fails to link due to __pioinfo apparently no longer being present in the new CRT (or else I'm linking the wrong libs?):
win32.obj : error LNK2001: unresolved external symbol __imp____pioinfo
This was added (actually restored and modified from earlier ripped out code) by:
http://perl5.git.perl.org/perl.git/commit/b47a847f6284f6f98ad7509cf77a4aeb802d8fce
As that commit notes\, "If __pioinfo isn't exported anymore\, the Perl build will break." So this was always known to be an area of danger and impending breakage; the question is: What do we do about it now that it's happened?
bulk88: Do you have any ideas on the latter point?
Oops\, spotted a couple of typos in win32.h in the v2 patch. Fixed in this new version.
Oops\, spotted a couple of typos in win32.h in the v2 patch. Fixed in this new version.
On Wed Jul 29 00:51:26 2015\, shay wrote:
This ticket is for centralizing discussions about how to add support for building with VS2015.
There have been a couple of p5p threads already; it is better to keep it all in one place:
http://www.nntp.perl.org/group/perl.perl5.porters/2015/05/msg227702.html http://www.nntp.perl.org/group/perl.perl5.porters/2015/07/msg229524.html
Attached is my first attempt\, originally posted in the latter thread above (saved here since http://www.nntp.perl.org doesn't seem to save attachments).
@@ -910\,6 +923\,17 @@ config.w32 : $(CFGSH_TMPL) @echo.>>$@ @echo #ifndef _config_h_footer_>>$@ @echo #define _config_h_footer_>>$@ +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" + @echo #undef FILE_ptr>>$@ + @echo #define FILE_ptr(fp) PERLIO_FILE_ptr(fp)>>$@ + @echo #undef FILE_cnt>>$@ + @echo #define FILE_cnt(fp) PERLIO_FILE_cnt(fp)>>$@ + @echo #undef FILE_base>>$@ + @echo #define FILE_base(fp) PERLIO_FILE_base(fp)>>$@ + @echo #undef FILE_bufsiz>>$@ + @echo #define FILE_bufsiz(fp) (PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))>>$@ + @echo #define I_STDBOOL>>$@ +!ENDIF @echo #undef Off_t>>$@ @echo #undef LSEEKSIZE>>$@ @echo #undef Off_t_size>>$@
This is overcomplicated. This should be done in win32.h #if _MSC_VER >= 1400. Why isn't PERLIO_FILE_base/etc used on every Win32 CC config including GCC and then PERLIO_FILE_base is implement one way or the other in win32.h? Those "+ @echo #undef FILE_ptr>>$@" lines are slow, each one is its own cmd.exe process launched by the make tool during the build process. It is best if there are less of them. This slowness note is from the Win32 parallel building project (see its ticket or prior ML threads).
@@ -563\,6 +567\,10 @@ CXX_FLAG = -TP -EHsc
LIBC = msvcrt.lib
+!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +LIBC = $(LIBC) vcruntime.lib ucrt.lib +!ENDIF +
Spiritually\, ucrt.lib/ucrtbase.dll is the MS libc now. VC 2015 msvcrt.lib only contains static link code for things like CPU probing and math ops. I am slightly slightly slightly afraid that lib won't be -e/-f-able anymore. This can break toolchain stuff\, like when you stuff double quotes to fix a spaces in path on cmd line problem in a file path to fix something EUMM/MB/CB-ish\, and now every breaks since it isn't a valid file path anymore. Maybe 2 of the 3 libs need to be put in $Config{libs} where oldnames.lib\, kernel32.lib\, user32.lib\, gdi32.lib\, etc are. I'd say ucrt.lib is the file to do probe for symbol names if you have some code that scans (not test compiles) "the libc" for what posix/c std lib symbols/functions it implements.
@@ -603\,6 +611\,11 @@ OPTIMIZE += -fp:precise DEFINES += -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE .ENDIF
+# Likewise for deprecated Winsock APIs in VC++ 14.0 for now. +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +DEFINES += -D_WINSOCK_DEPRECATED_NO_WARNINGS +.ENDIF + # In VS 2005 (VC++ 8.0) Microsoft changes time_t from 32-bit to # 64-bit\, even in 32-bit mode. It also provides the _USE_32BIT_TIME_T # preprocessor option to revert back to the old functionality for
There are now 3 huge "no deprecate" -Ds on the command line. There will probably be a new no-deprecate for every version of Visual C. These should really go into win32.h. Unix perl builds have atmost 2 or 3 -Ds\, not 10 or 15 -Ds. This might require reorganization of perl.h since the first CC .h is sys/types.h around line 670 in perl.h. config.h looks like the only place to the defines before sys/types.h. Some experiments need to be done if those no-deprecate defines can be moved to win32.h.
Also attached is my second attempt: This does surprisingly well with the quick-and-dirty approach of simply casting a FILE* to the appropriate new CRT struct (__crt_stdio_stream_data) and accessing the members of that. It might even suffice\, rather than going to the hassle of writing accessors and probably needing to separate getters from setters?
It looks like __crt_stdio_stream_data is allocated by the plain c calloc_base() function. That means msize can be used on it. msize is used for the pioinfo/osfhnd code for VC 2005 already since the struct repeatdly changed sizes during VC 2005's lifetime. On 32 bits\, __crt_stdio_stream_data is supposedly 0x38 bytes long\, IDK what it is on 64. You can write an assert/DEBUGGING comparing the msize() to sizeof() as a sanity check.
I support the casting+macro solution. It makes the best machine code.
It can also be argued that since VC 2015 is so "new"\, msize should always be used because an automatic OS update can change ucrtbase.dll at any time. IDK if ucrtbase.dll is VC redist package maintained (very rare to change)\, Service Pack/OS upgrade only changed (medium risk) or windows updates maintained (high risk of change). Unlike the osfhnd code\, where perl is only interested in the first member of the struct\, with __crt_stdio_stream_data it is interested in the middle members. If MS changes the order of the members under us\, we are screwed.
MS did change the order to move int _cnt between pointer "_ptr" and pointer "_base" to now sitting after _base http://www.nntp.perl.org/group/perl.perl5.porters/2015/05/msg227727.html when they created the UCRT. If there is a non-DEBUGGING/always msize done on a __crt_stdio_stream_data * at start up\, the only thing msize check will do is abort the perl process on startup\, since we can't know what the struct will be in the future. You will only get back a working perl by downgrading the OS/uninstall a hotfix\, or upgrade to the next perl maint release. At this point it is probably unlikely the struct members will be rearranged\, only new ones added to the end. That doesn't affect us in reaching the old members. My final opinion is the msize should be DEBUGGING only\, plus Ruby isn't doing it https://github.com/ruby/ruby/commit/3446537f1a8f6a0dbc2b27a0f25af13c7f61abf8 (IDK if Ruby has a ticket/discussion for that commit or not).
However\, the build currently fails to link due to __pioinfo apparently no longer being present in the new CRT (or else I'm linking the wrong libs?):
win32.obj : error LNK2001: unresolved external symbol __imp____pioinfo
This was added (actually restored and modified from earlier ripped out code) by:
http://perl5.git.perl.org/perl.git/commit/b47a847f6284f6f98ad7509cf77a4aeb802d8fce
As that commit notes\, "If __pioinfo isn't exported anymore\, the Perl build will break." So this was always known to be an area of danger and impending breakage; the question is: What do we do about it now that it's happened?
bulk88: Do you have any ideas on the latter point?
File a bug ticket with MS at https://connect.microsoft.com/VisualStudio/ to get __pioinfo from ucrtbase.dll.
In the mean time the easy but not best performance solution for is for Perl on VC 2015 to only support static UCRT from libucrt.lib since the libc/mscrt/ucrt symbols like _pioinfo can't hide from the linker when static linking.
Ruby has the same exact problem as we do https://github.com/ruby/ruby/blob/trunk/win32/win32.c#L2241 IDK how they are dealing with it or they are currently broken on VC 2015.
Old python used pioinfo https://github.com/python/cpython/blob/2.7/Modules/posixmodule.c#L532 they stopped using for VC 2015 https://github.com/python/cpython/blob/master/Modules/posixmodule.c#L1068 https://bugs.python.org/issue23524 \, I dont like their solution of swapping the callback everywhere. They were reaching inside the ioinfo struct to check if its allocated/open FD or not\, without triggering MS CRT's invalid param handler/exception/whatever. Python doesn't have the "sockets in libc FDs on win32" problem Perl and Ruby has\, python sockets live in a class by themselves so they aren't FDs https://rt-archive.perl.org/perl5/Ticket/Display.html?id=118127#txn-1357875 .
The other 2 solutions I have in my head on how to get ucrtbase.dll's _pioinfo are\, ummm\, crazy\, but would work. So any point in mentioning them or will I scare the children?
-- bulk88 ~ bulk88 at hotmail.com
The RT System itself - Status changed from 'new' to 'open'
This is a test case showing why the handle must be removed from the CRT fd before calling close. This test case is sorta intended to be given to MS in a ticket.
-- bulk88 ~ bulk88 at hotmail.com
ntdll.dll!_KiRaiseUserExceptionDispatcher@0() + 0x37
ntdll.dll!_KiFastSystemCall@0() + 0x3
ntdll.dll!_NtClose@4() + 0xc
mswsock.dll!_WSPCloseSocket@8() + 0x129
ws2_32.dll!_closesocket@4() + 0x47
closeclosesocket.exe!main(int argc=1\, char * * argv=0x00391040) Line 22 C++
closeclosesocket.exe!mainCRTStartup() Line 259 + 0x12 C
kernel32.dll!_BaseProcessStart@4() + 0x23
"Time of Day"\,"Process Name"\,"PID"\,"TID"\,"Operation"\,"Path"\,"Result"\,"Detail"
"1:42:00.3590502 PM"\,"closeclosesocket.exe"\,"4552"\,"6140"\,"RegCloseKey"\,"\
Unhandled exception at 0x7c90e4ff (ntdll.dll) in closesocketclose.exe: 0xC0000008: An invalid HANDLE was specified.
ntdll.dll!_KiRaiseUserExceptionDispatcher@0() + 0x37
ntdll.dll!_KiFastSystemCall@0() + 0x3
ntdll.dll!_NtClose@4() + 0xc
kernel32.dll!_CloseHandle@4() + 0x44
closesocketclose.exe!_close(int fh=3) Line 115 + 0x3a C
closesocketclose.exe!main(int argc=1\, char * * argv=0x00391040) Line 21 + 0xc C++
closesocketclose.exe!mainCRTStartup() Line 259 + 0x12 C
kernel32.dll!_BaseProcessStart@4() + 0x23
"Time of Day"\,"Process Name"\,"PID"\,"TID"\,"Operation"\,"Path"\,"Result"\,"Detail"
"1:35:14.4713377 PM"\,"closesocketclose.exe"\,"2636"\,"4356"\,"RegCloseKey"\,"\
On 29 July 2015 at 13:39\, bulk88 via RT \perlbug\-followup@​perl\.org wrote:
On Wed Jul 29 00:51:26 2015\, shay wrote: ---------------------------- @@ -1074\,6 +1087\,17 @@ config.w32 : $(CFGSH_TMPL) @echo.>>$@ @echo #ifndef _config_h_footer_>>$@ @echo #define _config_h_footer_>>$@ +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" + @echo #undef FILE_ptr>>$@ + @echo #define FILE_ptr(fp) PERLIO_FILE_ptr(fp)>>$@ + @echo #undef FILE_cnt>>$@ + @echo #define FILE_cnt(fp) PERLIO_FILE_cnt(fp)>>$@ + @echo #undef FILE_base>>$@ + @echo #define FILE_base(fp) PERLIO_FILE_base(fp)>>$@ + @echo #undef FILE_bufsiz>>$@ + @echo #define FILE_bufsiz(fp) (PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))>>$@ + @echo #define I_STDBOOL>>$@ +.ENDIF @echo #undef Off_t>>$@ @echo #undef LSEEKSIZE>>$@ @echo #undef Off_t_size>>$@ ---------------------------------------------- This is overcomplicated. This should be done in win32.h #if _MSC_VER >= 1400. Why isn't PERLIO_FILE_base/etc used on every Win32 CC config including GCC and then PERLIO_FILE_base is implement one way or the other in win32.h? Those "+ @echo #undef FILE_ptr>>$@" lines are slow\, each one is its own cmd.exe process launched by the make tool during the build process. It is best if there are less of them. This slowness note is from the Win32 parallel building project (see its ticket or prior ML threads).
I will come back to this later since it's a separate issue from getting things building with VS2015. The whole business of all these echo lines needs sorting out better somehow\, not just these new lines.
---------------------------------------------- @@ -563\,6 +567\,10 @@ CXX_FLAG = -TP -EHsc
LIBC = msvcrt.lib
+!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +LIBC = $(LIBC) vcruntime.lib ucrt.lib +!ENDIF + ----------------------------------------------
Spiritually\, ucrt.lib/ucrtbase.dll is the MS libc now. VC 2015 msvcrt.lib only contains static link code for things like CPU probing and math ops. I am slightly slightly slightly afraid that lib won't be -e/-f-able anymore. This can break toolchain stuff\, like when you stuff double quotes to fix a spaces in path on cmd line problem in a file path to fix something EUMM/MB/CB-ish\, and now every breaks since it isn't a valid file path anymore. Maybe 2 of the 3 libs need to be put in $Config{libs} where oldnames.lib\, kernel32.lib\, user32.lib\, gdi32.lib\, etc are. I'd say ucrt.lib is the file to do probe for symbol names if you have some code that scans (not test compiles) "the libc" for what posix/c std lib symbols/functions it implements.
Done in the v3 patch attached.
------------------------------------------------ @@ -603\,6 +611\,11 @@ OPTIMIZE += -fp:precise DEFINES += -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE .ENDIF
+# Likewise for deprecated Winsock APIs in VC++ 14.0 for now. +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +DEFINES += -D_WINSOCK_DEPRECATED_NO_WARNINGS +.ENDIF + # In VS 2005 (VC++ 8.0) Microsoft changes time_t from 32-bit to # 64-bit\, even in 32-bit mode. It also provides the _USE_32BIT_TIME_T # preprocessor option to revert back to the old functionality for ------------------------------------------------
There are now 3 huge "no deprecate" -Ds on the command line. There will probably be a new no-deprecate for every version of Visual C. These should really go into win32.h. Unix perl builds have atmost 2 or 3 -Ds\, not 10 or 15 -Ds. This might require reorganization of perl.h since the first CC .h is sys/types.h around line 670 in perl.h. config.h looks like the only place to the defines before sys/types.h. Some experiments need to be done if those no-deprecate defines can be moved to win32.h.
I will also return to this since it is again separate from VS2015 support.
Also attached is my second attempt: This does surprisingly well with the quick-and-dirty approach of simply casting a FILE* to the appropriate new CRT struct (__crt_stdio_stream_data) and accessing the members of that. It might even suffice\, rather than going to the hassle of writing accessors and probably needing to separate getters from setters?
It looks like __crt_stdio_stream_data is allocated by the plain c calloc_base() function. That means msize can be used on it. msize is used for the pioinfo/osfhnd code for VC 2005 already since the struct repeatdly changed sizes during VC 2005's lifetime. On 32 bits\, __crt_stdio_stream_data is supposedly 0x38 bytes long\, IDK what it is on 64. You can write an assert/DEBUGGING comparing the msize() to sizeof() as a sanity check.
I support the casting+macro solution. It makes the best machine code.
It can also be argued that since VC 2015 is so "new"\, msize should always be used because an automatic OS update can change ucrtbase.dll at any time. IDK if ucrtbase.dll is VC redist package maintained (very rare to change)\, Service Pack/OS upgrade only changed (medium risk) or windows updates maintained (high risk of change). Unlike the osfhnd code\, where perl is only interested in the first member of the struct\, with __crt_stdio_stream_data it is interested in the middle members. If MS changes the order of the members under us\, we are screwed.
MS did change the order to move int _cnt between pointer "_ptr" and pointer "_base" to now sitting after _base http://www.nntp.perl.org/group/perl.perl5.porters/2015/05/msg227727.html when they created the UCRT. If there is a non-DEBUGGING/always msize done on a __crt_stdio_stream_data * at start up\, the only thing msize check will do is abort the perl process on startup\, since we can't know what the struct will be in the future. You will only get back a working perl by downgrading the OS/uninstall a hotfix\, or upgrade to the next perl maint release. At this point it is probably unlikely the struct members will be rearranged\, only new ones added to the end. That doesn't affect us in reaching the old members. My final opinion is the msize should be DEBUGGING only\, plus Ruby isn't doing it https://github.com/ruby/ruby/commit/3446537f1a8f6a0dbc2b27a0f25af13c7f61abf8 (IDK if Ruby has a ticket/discussion for that commit or not).
Done in the v3 patch attached.
However\, the build currently fails to link due to __pioinfo apparently no longer being present in the new CRT (or else I'm linking the wrong libs?):
win32.obj : error LNK2001: unresolved external symbol __imp____pioinfo
This was added (actually restored and modified from earlier ripped out code) by:
http://perl5.git.perl.org/perl.git/commit/b47a847f6284f6f98ad7509cf77a4aeb802d8fce
As that commit notes\, "If __pioinfo isn't exported anymore\, the Perl build will break." So this was always known to be an area of danger and impending breakage; the question is: What do we do about it now that it's happened?
bulk88: Do you have any ideas on the latter point?
File a bug ticket with MS at https://connect.microsoft.com/VisualStudio/ to get __pioinfo from ucrtbase.dll.
In the mean time the easy but not best performance solution for is for Perl on VC 2015 to only support static UCRT from libucrt.lib since the libc/mscrt/ucrt symbols like _pioinfo can't hide from the linker when static linking.
Done in the v3 patch attached\, but I'm not very keen on this. Perl has always used the CRT DLL\, which makes me very uneasy about switching to LIBC. It does get it building\, though\, but now crashes on startup trying to set environ[0] to NULL in S_init_postdump_symbols().
I briefly tried #defining NO_ENVIRON_ARRAY to temporarily duck that issue but the build failed because Perl_my_setenv() is then not provided by util.c\, but is apparently still required by mg.c (at least)... :-/
Is this problem due to the switch from dynamic to static CRT? Or just something new in VC14 anyway?
Ruby has the same exact problem as we do https://github.com/ruby/ruby/blob/trunk/win32/win32.c#L2241 IDK how they are dealing with it or they are currently broken on VC 2015.
Old python used pioinfo https://github.com/python/cpython/blob/2.7/Modules/posixmodule.c#L532 they stopped using for VC 2015 https://github.com/python/cpython/blob/master/Modules/posixmodule.c#L1068 https://bugs.python.org/issue23524 \, I dont like their solution of swapping the callback everywhere. They were reaching inside the ioinfo struct to check if its allocated/open FD or not\, without triggering MS CRT's invalid param handler/exception/whatever. Python doesn't have the "sockets in libc FDs on win32" problem Perl and Ruby has\, python sockets live in a class by themselves so they aren't FDs https://rt-archive.perl.org/perl5/Ticket/Display.html?id=118127#txn-1357875 .
The other 2 solutions I have in my head on how to get ucrtbase.dll's _pioinfo are\, ummm\, crazy\, but would work. So any point in mentioning them or will I scare the children?
Since I'm not keen on the static CRT change\, what are the other options that you see here?
On Thu Jul 30 11:10:57 2015\, Steve.Hay@verosoftware.com wrote:
Since I'm not keen on the static CRT change\, what are the other options that you see here?
Can you send me your ucrtbase.dll files offlist\, 32 and 64 bit ones. Also are you building 64 or 32 bit perl with 2015?
As a side note\, here is an interesting link http://blogs.msdn.com/b/oldnewthing/archive/2014/04/11/10516280.aspx .
Here is a ruby ticket with MS about pioinfo https://connect.microsoft.com/VisualStudio/feedback/details/1279133
-- bulk88 ~ bulk88 at hotmail.com
On Thu Jul 30 11:10:57 2015\, Steve.Hay@verosoftware.com wrote:
Done in the v3 patch attached\, but I'm not very keen on this. Perl has always used the CRT DLL\, which makes me very uneasy about switching to LIBC. It does get it building\, though\, but now crashes on startup trying to set environ[0] to NULL in S_init_postdump_symbols().
I briefly tried #defining NO_ENVIRON_ARRAY to temporarily duck that issue but the build failed because Perl_my_setenv() is then not provided by util.c\, but is apparently still required by mg.c (at least)... :-/
The environ deref NULL SEGV is fixed by the attached patch\, which goes ontop of your patch "Add VS2015 support\, v3"\, you can rebase squash my patch with yours once you fine with it.
Basically either all 3 .libs are the static ones\, or the DLL ones. No mixing allowed.
The actual crash was environ var in static CRT was uninitialized because this function from vcruntime wasn't running when initterm_e when through its array of C-MS-extensions/C++ global initializer functions.
_CRTALLOC(".CRT$XIAA") static _PIFV pre_c_initializer = pre_c_initialization;
************cut************ static int __cdecl pre_c_initialization() throw() { _set_app_type(main_policy::get_app_type());
************cut************ initialize_environment();
__scrt_initialize_winrt();
return 0; }
With my patch\, there are no assert fails with DEBUGGING (I think\, I can a couple tests manually they were find)\, but harness is very broken. I am using the 32 bit RC VC 2015 from April 2015\, so try it on your machine and see if you are getting this
perl.exe harness base/cond.t ....................................................... No subtests run base/if.t ......................................................... No subtests run base/lex.t ........................................................ No subtests run base/num.t ........................................................ No subtests run base/pat.t ........................................................ No subtests run base/rs.t ......................................................... No subtests run base/term.t ....................................................... No subtests
IDK where the problem is since backticks work
C:\p521\src>perl -MData::Dumper -E"say Dumper(scalar(`$^X -E\"say 'hi'\"`))" $VAR1 = 'hi ';
I am going to now work on getting the DLL UCRT working instead of this static CRT build.
-- bulk88 ~ bulk88 at hotmail.com
On Thu Jul 30 11:10:57 2015\, Steve.Hay@verosoftware.com wrote:
Since I'm not keen on the static CRT change\, what are the other options that you see here?
There are more options/choices than the patch I've attached\, but this patch was the easiest to create of all the choices.
On Sat Aug 01 01:25:57 2015\, bulk88 wrote:
I am going to now work on getting the DLL UCRT working instead of this static CRT build.
I've attached a patch for the easiest solution to no more __pioinfo export. The patch only works on 32 bit perl builds with ucrtbase.dll\, 64 bits and any ucrtbased.dll are not supported ATM.
Smoking the perl gave me
Test Summary Report
../cpan/IPC-Cmd/t/01_IPC-Cmd.t (Wstat: 768 Test s: 459 Failed: 3) Failed tests: 38-39\, 64 Non-zero exit status: 3 ../cpan/parent/t/parent-pmc.t (Wstat: 768 Test s: 3 Failed: 3) Failed tests: 1-3 Non-zero exit status: 3 ../dist/Carp/t/errno.t (Wstat: 1024 Tes ts: 20 Failed: 4) Failed tests: 9\, 12\, 16\, 20 Non-zero exit status: 4 Files=2391\, Tests=710888\, 10548 wallclock secs (76.30 usr + 4.90 sys = 81.20 CP U) Result: FAIL NMAKE : fatal error U1077: '.\perl.exe' : return code '0xa' Stop.
../cpan/parent/t/parent-pmc.t can be ignored
../dist/Carp/t/errno.t failure is unexpected\, maybe it is because of my old RC VC 2015
C:\p521\src\t>perl harness -v ../dist/Carp/t/errno.t ../dist/Carp/t/errno.t .. 1..20 ok 1 ok 2 ok 3 ok 4 ok 5 ok 6 ok 7 ok 8
not ok 9 # Failed test at t/errno.t line 38. # got: '0' # expected: '69' ok 10 ok 11
not ok 12# Failed test at t/errno.t line 46.
# got: '0' # expected: '69' ok 13 ok 14 ok 15
# Failed test at t/errno.t line 58. not ok 16 # got: '0' # expected: '69' ok 17 ok 18 ok 19
not ok 20 # Failed test at t/errno.t line 67. # got: '0' # expected: '69' # Looks like you failed 4 tests of 20. Dubious\, test returned 4 (wstat 1024\, 0x400) Failed 4/20 subtests
Test Summary Report
../dist/Carp/t/errno.t (Wstat: 1024 Tests: 20 Failed: 4) Failed tests: 9\, 12\, 16\, 20 Non-zero exit status: 4 Files=1\, Tests=20\, 0 wallclock secs ( 0.02 usr + 0.00 sys = 0.02 CPU) Result: FAIL
C:\p521\src\t>
from reading errno.t\, it looks like something new is calling SetLastError somewhere. Ill have to set some BPs and find out tomorrow.
-- bulk88 ~ bulk88 at hotmail.com
On Sat Aug 01 01:25:57 2015\, bulk88 wrote:
On Thu Jul 30 11:10:57 2015\, Steve.Hay@verosoftware.com wrote:
Done in the v3 patch attached\, but I'm not very keen on this. Perl has always used the CRT DLL\, which makes me very uneasy about switching to LIBC. It does get it building\, though\, but now crashes on startup trying to set environ[0] to NULL in S_init_postdump_symbols().
I briefly tried #defining NO_ENVIRON_ARRAY to temporarily duck that issue but the build failed because Perl_my_setenv() is then not provided by util.c\, but is apparently still required by mg.c (at least)... :-/
The environ deref NULL SEGV is fixed by the attached patch\, which goes ontop of your patch "Add VS2015 support\, v3"\, you can rebase squash my patch with yours once you fine with it.
Basically either all 3 .libs are the static ones\, or the DLL ones. No mixing allowed.
The actual crash was environ var in static CRT was uninitialized because this function from vcruntime wasn't running when initterm_e when through its array of C-MS-extensions/C++ global initializer functions.
---------------------------------------------------------------- _CRTALLOC(".CRT$XIAA") static _PIFV pre_c_initializer = pre_c_initialization;
************cut************ static int __cdecl pre_c_initialization() throw() { _set_app_type(main_policy::get_app_type());
************cut************ initialize_environment();
__scrt_initialize_winrt();
return 0; } ----------------------------------------------------------------
With my patch\, there are no assert fails with DEBUGGING (I think\, I can a couple tests manually they were find)\, but harness is very broken. I am using the 32 bit RC VC 2015 from April 2015\, so try it on your machine and see if you are getting this
------------------------------------------------------------------------------- perl.exe harness base/cond.t ....................................................... No subtests run base/if.t ......................................................... No subtests run base/lex.t ........................................................ No subtests run base/num.t ........................................................ No subtests run base/pat.t ........................................................ No subtests run base/rs.t ......................................................... No subtests run base/term.t ....................................................... No subtests -------------------------------------------------------------------------------
IDK where the problem is since backticks work ----------------------------------------------------------- C:\p521\src>perl -MData::Dumper -E"say Dumper(scalar(`$^X -E\"say 'hi'\"`))" $VAR1 = 'hi '; -----------------------------------------------------------
I am going to now work on getting the DLL UCRT working instead of this static CRT build.
Thanks for the environ fix and your other changes. I've squashed them together with mine in the attached v4 patch.
With this\, I got a bunch of test failures\, very different from your report\, and most not reproducible when I re-ran them. But three are reproducible failures:
C:\Dev\Git\perl\t>.\perl harness run/locale.t run/locale.t .. 3/37 # Failed test 8 - localeconv() looks at LC_NUMERIC with and without 'use locale' at ./test.pl line 1027 # got ".." # expected "\,\," # PROG: # print localeconv()->{decimal_point}; # use POSIX; # use locale; # print localeconv()->{decimal_point}; # STATUS: 0 run/locale.t .. Failed 1/37 subtests (less 6 skipped subtests: 30 okay)
Test Summary Report
run/locale.t (Wstat: 0 Tests: 37 Failed: 1) Failed test: 8 Files=1\, Tests=37\, 4 wallclock secs ( 0.05 usr + 0.00 sys = 0.05 CPU) Result: FAIL
C:\Dev\Git\perl\t>.\perl harness ../ext/PerlIO-scalar/t/scalar.t ../ext/PerlIO-scalar/t/scalar.t .. 1/122 # Failed test 'check $! is updated' # at t/scalar.t line 406. # got: '0' # expected: '22'
# Failed test 'check $! is updated even when we warn' # at t/scalar.t line 411. # got: '0' # expected: '22'
# Failed test 'check errno set correctly' # at t/scalar.t line 444. # got: '0' # expected: '22'
# Failed test 'check errno set' # at t/scalar.t line 466. # got: '0' # expected: '22' # Looks like you failed 4 tests of 122. ../ext/PerlIO-scalar/t/scalar.t .. Dubious\, test returned 4 (wstat 1024\, 0x400) Failed 4/122 subtests
Test Summary Report
../ext/PerlIO-scalar/t/scalar.t (Wstat: 1024 Tests: 122 Failed: 4) Failed tests: 86\, 89\, 100\, 107 Non-zero exit status: 4 Files=1\, Tests=122\, 0 wallclock secs ( 0.02 usr + 0.02 sys = 0.03 CPU) Result: FAIL
C:\Dev\Git\perl\t>.\perl harness ../ext/XS-Typemap/t/Typemap.t ../ext/XS-Typemap/t/Typemap.t .. 1/156 # Failed test at t/Typemap.t line 367. Label not found for "last SKIP" at ../../lib/Test/More.pm line 1314. # Looks like you planned 156 tests but ran 134. # Looks like you failed 1 test of 134 run. # Looks like your test exited with 9 just after 134. ../ext/XS-Typemap/t/Typemap.t .. Dubious\, test returned 9 (wstat 2304\, 0x900) Failed 23/156 subtests
Test Summary Report
../ext/XS-Typemap/t/Typemap.t (Wstat: 2304 Tests: 134 Failed: 1) Failed test: 134 Non-zero exit status: 9 Parse errors: Bad plan. You planned 156 tests but ran 134. Files=1\, Tests=134\, 0 wallclock secs ( 0.03 usr + 0.00 sys = 0.03 CPU) Result: FAIL
I will look at these in more detail soon\, but I'm inclined to commit what we've got so far so we've got something definite to base further efforts on\, and so that other people trying VS2015 won't fail at the first hurdle and waste time duplicating our work here.
I also want to return to the other comments that you made which I said I'd come back to\, and I haven't tried out your solution to avoid using the static CRT libs yet... I had a brief look\, and yes it is enough to scare the children! ;-) It makes it look like perl is doing something really naughty to have to go to such lengths. I think it would be worth submitting your other attachments (from https://rt-archive.perl.org/perl5/Ticket/Display.html?id=125714#txn-1358960) regarding that to MS as a bug report to see if there is some better way of doing this. Do you want to do that or shall I?
On Tue Aug 04 01:07:15 2015\, shay wrote:
Thanks for the environ fix and your other changes. I've squashed them together with mine in the attached v4 patch.
With this\, I got a bunch of test failures\, very different from your report\, and most not reproducible when I re-ran them. But three are reproducible failures:
C:\Dev\Git\perl\t>.\perl harness run/locale.t run/locale.t .. 3/37 # Failed test 8 - localeconv() looks at LC_NUMERIC with and without 'use locale' at ./test.pl line 1027 # got ".." # expected "\,\," # PROG: # print localeconv()->{decimal_point}; # use POSIX; # use locale; # print localeconv()->{decimal_point}; # STATUS: 0 run/locale.t .. Failed 1/37 subtests (less 6 skipped subtests: 30 okay)
Test Summary Report ------------------- run/locale.t (Wstat: 0 Tests: 37 Failed: 1) Failed test: 8 Files=1\, Tests=37\, 4 wallclock secs ( 0.05 usr + 0.00 sys = 0.05 CPU) Result: FAIL
C:\Dev\Git\perl\t>.\perl harness ../ext/PerlIO-scalar/t/scalar.t ../ext/PerlIO-scalar/t/scalar.t .. 1/122 # Failed test 'check $! is updated' # at t/scalar.t line 406. # got: '0' # expected: '22'
# Failed test 'check $! is updated even when we warn' # at t/scalar.t line 411. # got: '0' # expected: '22'
# Failed test 'check errno set correctly' # at t/scalar.t line 444. # got: '0' # expected: '22'
# Failed test 'check errno set' # at t/scalar.t line 466. # got: '0' # expected: '22' # Looks like you failed 4 tests of 122. ../ext/PerlIO-scalar/t/scalar.t .. Dubious\, test returned 4 (wstat 1024\, 0x400) Failed 4/122 subtests
Test Summary Report ------------------- ../ext/PerlIO-scalar/t/scalar.t (Wstat: 1024 Tests: 122 Failed: 4) Failed tests: 86\, 89\, 100\, 107 Non-zero exit status: 4 Files=1\, Tests=122\, 0 wallclock secs ( 0.02 usr + 0.02 sys = 0.03 CPU) Result: FAIL
In my RC 2015 build\, the ../dist/Carp/t/errno.t failures I am seeing is because there is an unprotect by save/restoring GLR FlsGetValue call from Perl_my_snprintf from _LocaleUpdate inside the CRT. Maybe this case was fixed by MS for gold release\, but other cases of FlsGetValue are around in the UCRT.
KernelBase.dll!_FlsGetValue@4() Unknown ucrtbase.dll!___acrt_FlsGetValue@4() Unknown ucrtbase.dll!___acrt_update_locale_info() Unknown ucrtbase.dll!_LocaleUpdate::_LocaleUpdate() Unknown ucrtbase.dll!common_vsprintf\<__crt_stdio_output::standard_base\,char>() Unknown ucrtbase.dll!___stdio_common_vsprintf() Unknown perl523.dll!_vsnprintf_l(char * const _Buffer\, const unsigned int _BufferCount\, const char * const _Format\, __crt_locale_pointers * const _ArgList\, char *) Line 1390 C perl523.dll!Perl_my_snprintf(char * buffer\, const unsigned int len\, const char * format\, ...) Line 5137 C perl523.dll!Perl_save_re_context(interpreter * my_perl) Line 17781 C perl523.dll!S_invoke_exception_hook(interpreter * my_perl\, sv * ex\, bool warn) Line 1552 C perl523.dll!Perl_warn_sv(interpreter * my_perl\, sv * baseex) Line 1834 C perl523.dll!Perl_pp_warn(interpreter * my_perl) Line 462 C perl523.dll!Perl_runops_standard(interpreter * my_perl) Line 41 C perl523.dll!S_run_body(interpreter * my_perl\, long oldscope) Line 2448 C perl523.dll!perl_run(interpreter * my_perl) Line 2371 C perl523.dll!RunPerl(int argc\, char * * argv\, char * * env) Line 258 C++ [External Code]
C:\Dev\Git\perl\t>.\perl harness ../ext/XS-Typemap/t/Typemap.t ../ext/XS-Typemap/t/Typemap.t .. 1/156 # Failed test at t/Typemap.t line 367. Label not found for "last SKIP" at ../../lib/Test/More.pm line 1314. # Looks like you planned 156 tests but ran 134. # Looks like you failed 1 test of 134 run. # Looks like your test exited with 9 just after 134. ../ext/XS-Typemap/t/Typemap.t .. Dubious\, test returned 9 (wstat 2304\, 0x900) Failed 23/156 subtests
Test Summary Report ------------------- ../ext/XS-Typemap/t/Typemap.t (Wstat: 2304 Tests: 134 Failed: 1) Failed test: 134 Non-zero exit status: 9 Parse errors: Bad plan. You planned 156 tests but ran 134. Files=1\, Tests=134\, 0 wallclock secs ( 0.03 usr + 0.00 sys = 0.03 CPU) Result: FAIL
Line 367
is
# T_STDIO note("T_STDIO");
# open a file in XS for write my $testfile= "stdio.tmp"; # not everything below cleans up END { 1 while unlink $testfile; } my $fh = T_STDIO_open( $testfile ); ok( $fh );\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<
# write to it using perl if (defined $fh) {
Now everyone remember\, Win32 Perl has NEVER been compiled with static CRT before. For any VC version. I should try a VC 2003 static CRT build and see how many problems I get.
In my DLL CRT build with the pioinfo patch\, in Typemap.dll _Perl_newXS_deffile(my_perl\, "XS::Typemap::T_STDIO_open"\, XS_XS__Typemap_T_STDIO_open);
and inside XS_XS__Typemap_T_STDIO_open I see
v8 = v6->sv_u.sv_flags; if ( (v8 & 0x200400) == 1024 ) v9 = v6->sv_u.svu_pv; else v9 = (const char *)_Perl_sv_2pv_flags(my_perl\, v6\, 0\, 2); v12 = _fopen(v9\, "w"); v15 = _Perl_sv_newmortal(my_perl); v11 = _Perl_newGVgen_flags(v3\, "XS::Typemap"\, 0); v10 = _PerlIO_importFILE(v12\, 0);
the _fopen symbol is imported from api-ms-win-crt-stdio-l1-1-0.dll which is really ucrtbase.dll.
In a static CRT the fds are isolated to each DLL. Win32 perl has xsub.h and all its c std lib hooking macros that redirect to win32_* functions. In DLL CRT perl win32_* are jump stubs to CRT DLL functions\, in static CRT perl the win32_* goto the internal static linked CRT in perl523.dll. perl.exe's copy of the CRT seems to play almost no role in perl interp\, since the whole interp lives in perl523.dll and i think perl.exe's CRT irrelevant. In this case\, XS-Typemap intentionally bypasses xsub.h in its testing
XS_EUPXS(XS_XS__Typemap_T_STDIO_open); /* prototype to pass -Wmissing-prototypes */ XS_EUPXS(XS_XS__Typemap_T_STDIO_open) { dVAR; dXSARGS; if (items != 1) croak_xs_usage(cv\, "file"); { const char * file = (const char *)SvPV_nolen(ST(0)) ; FILE * RETVAL; #line 905 "Typemap.xs" RETVAL = xsfopen( file );\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\< #line 1694 "Typemap.c" { SV * RETVALSV; RETVALSV = sv_newmortal(); { GV *gv = newGVgen("XS::Typemap"); PerlIO *fp = PerlIO_importFILE(RETVAL\,0); if ( fp && do_open(gv\, "+\<&"\, 3\, FALSE\, 0\, 0\, fp) ) { SV *rv = newRV_inc((SV*)gv); rv = sv_bless(rv\, GvSTASH(gv)); RETVALSV = sv_2mortal(rv); } } ST(0) = RETVALSV; } } XSRETURN(1); }
http://perl5.git.perl.org/perl.git/blob/HEAD:/ext/XS-Typemap/stdio.c
/* This provides a test of STDIO and emulates a library that has been built outside of the PerlIO system and therefore is built using FILE* rather than PerlIO * (a common occurrence for XS).
Use a separate file to make sure we are not contaminated by PerlIO. */
#include \<stdio.h>
/* Open a file for write */ FILE * xsfopen ( const char * path ) { FILE * stream; stream = fopen( path\, "w"); return stream; }
IDK about making the test use win32_fopen instead of CRT fopen since that is violating the spirit of XS-Typemap's tests. Maybe a skip if $Config{win32_static_CRT} in ../ext/XS-Typemap/t/Typemap.t around that test?
I will look at these in more detail soon\, but I'm inclined to commit what we've got so far so we've got something definite to base further efforts on\, and so that other people trying VS2015 won't fail at the first hurdle and waste time duplicating our work here.
Other people? non-C coders building win32 perl from source or TonyC/JDB?
I'd add a warning to the 2 makefiles at
@@ -126\,6 +126\,10 @@ CCTYPE = MSVC60 #CCTYPE = MSVC120 # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free version) #CCTYPE = MSVC120FREE +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) +#CCTYPE = MSVC140 +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free version) +#CCTYPE = MSVC140FREE
stating "work in progress" or "not supported" and an echo in
.\config.h : $(CFGH_TMPL) -del /f config.h copy $(CFGH_TMPL) config.h @echo.>>$@
or on another VC 2015 (through ifdefs) specific target an echo warning saying "your VC 2015 is experimental". Can't have random people thinking it works 100%.
I would also rename LINK_FLAG to LIBC_FLAG or CRT_FLAG or RTL_FLAG (is it the CRT or RTL MS? make up your mind). LINK_FLAG is too generic of a term. There is already a macro called LINK_FLAGS. LINK_FLAG vs LINK_FLAGS is confusing. Also LINK_FLAG is being passed to cl.exe\, not link.exe\, which is more confusing.
-- bulk88 ~ bulk88 at hotmail.com
On 5 August 2015 at 05:06\, bulk88 via RT \perlbug\-followup@​perl\.org wrote:
Now everyone remember\, Win32 Perl has NEVER been compiled with static CRT before. For any VC version. I should try a VC 2003 static CRT build and see how many problems I get.
Good point. I should have thought of that myself: I've now tried VS2010 using /MT for the static libcmt.lib and I get the *exact* same pattern of test failures\, so the failures are surely due to using the static CRT\, not due to changes in VS2015. They're therefore probably not worth pursuing any further right now since I hope we can revert to the dynamic CRT once we're settled on the best (least scariest?!) way to do that\, possibly after quizzing MS on our problems.
I will look at these in more detail soon\, but I'm inclined to commit what we've got so far so we've got something definite to base further efforts on\, and so that other people trying VS2015 won't fail at the first hurdle and waste time duplicating our work here.
Other people? non-C coders building win32 perl from source or TonyC/JDB?
I'd add a warning to the 2 makefiles at
@@ -126\,6 +126\,10 @@ CCTYPE = MSVC60 #CCTYPE = MSVC120 # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free version) #CCTYPE = MSVC120FREE +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) +#CCTYPE = MSVC140 +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free version) +#CCTYPE = MSVC140FREE
stating "work in progress" or "not supported" and an echo in
.\config.h : $(CFGH_TMPL) -del /f config.h copy $(CFGH_TMPL) config.h @echo.>>$@
or on another VC 2015 (through ifdefs) specific target an echo warning saying "your VC 2015 is experimental". Can't have random people thinking it works 100%.
Yes\, I've added suitable warnings into the makefiles and the README.win32. I will test a few more configurations today and hopefully commit later.
I would also rename LINK_FLAG to LIBC_FLAG or CRT_FLAG or RTL_FLAG (is it the CRT or RTL MS? make up your mind). LINK_FLAG is too generic of a term. There is already a macro called LINK_FLAGS. LINK_FLAG vs LINK_FLAGS is confusing. Also LINK_FLAG is being passed to cl.exe\, not link.exe\, which is more confusing.
Agreed. Renamed to LIBC_FLAG.
On Wed Aug 05 01:25:36 2015\, Steve.Hay@verosoftware.com wrote:
I will look at these in more detail soon\, but I'm inclined to commit what we've got so far so we've got something definite to base further efforts on\, and so that other people trying VS2015 won't fail at the first hurdle and waste time duplicating our work here.
Other people? non-C coders building win32 perl from source or TonyC/JDB?
I'd add a warning to the 2 makefiles at
@@ -126\,6 +126\,10 @@ CCTYPE = MSVC60 #CCTYPE = MSVC120 # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free version) #CCTYPE = MSVC120FREE +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) +#CCTYPE = MSVC140 +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free version) +#CCTYPE = MSVC140FREE
stating "work in progress" or "not supported" and an echo in
.\config.h : $(CFGH_TMPL) -del /f config.h copy $(CFGH_TMPL) config.h @echo.>>$@
or on another VC 2015 (through ifdefs) specific target an echo warning saying "your VC 2015 is experimental". Can't have random people thinking it works 100%.
Yes\, I've added suitable warnings into the makefiles and the README.win32. I will test a few more configurations today and hopefully commit later.
I rebuilt with release VC 2015\, and your V4 patch (static CRT)\, I am still seeing
perl.exe harness base/cond.t ....................................................... No subtests run base/if.t ......................................................... No subtests run base/lex.t ........................................................ No subtests run base/num.t ........................................................ No subtests run base/pat.t ........................................................ No subtests run base/rs.t ......................................................... No subtests run base/term.t ....................................................... No subtests
for every test.
I did BUILD_STATIC and ALL_STATIC build\, with fat bundled XS perl523.dll I still get "No subtests run" from harness. Only when I deleted /t/perl.exe and perl.exe and renamed perl-static.exe to perl.exe and copied fat (static CRT+static XS) perl.exe to t/perl.exe did I get normal harness running and the "No subtests run" failures went away. Changing blead (no steveh vc 2015 patches) to static CRT with VC 2003 also causes "No subtests run". Are you seeing harness fail like this? Is there something special you are doing?
-- bulk88 ~ bulk88 at hotmail.com
On 6 August 2015 at 03:41\, bulk88 via RT \perlbug\-followup@​perl\.org wrote:
On Wed Aug 05 01:25:36 2015\, Steve.Hay@verosoftware.com wrote:
I will look at these in more detail soon\, but I'm inclined to commit what we've got so far so we've got something definite to base further efforts on\, and so that other people trying VS2015 won't fail at the first hurdle and waste time duplicating our work here.
Other people? non-C coders building win32 perl from source or TonyC/JDB?
I'd add a warning to the 2 makefiles at
@@ -126\,6 +126\,10 @@ CCTYPE = MSVC60 #CCTYPE = MSVC120 # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free version) #CCTYPE = MSVC120FREE +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) +#CCTYPE = MSVC140 +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free version) +#CCTYPE = MSVC140FREE
stating "work in progress" or "not supported" and an echo in
.\config.h : $(CFGH_TMPL) -del /f config.h copy $(CFGH_TMPL) config.h @echo.>>$@
or on another VC 2015 (through ifdefs) specific target an echo warning saying "your VC 2015 is experimental". Can't have random people thinking it works 100%.
Yes\, I've added suitable warnings into the makefiles and the README.win32. I will test a few more configurations today and hopefully commit later.
I rebuilt with release VC 2015\, and your V4 patch (static CRT)\, I am still seeing ------------------------------------------------------------------------------- perl.exe harness base/cond.t ....................................................... No subtests run base/if.t ......................................................... No subtests run base/lex.t ........................................................ No subtests run base/num.t ........................................................ No subtests run base/pat.t ........................................................ No subtests run base/rs.t ......................................................... No subtests run base/term.t ....................................................... No subtests ------------------------------------------------------------------------------- for every test.
I did BUILD_STATIC and ALL_STATIC build\, with fat bundled XS perl523.dll I still get "No subtests run" from harness. Only when I deleted /t/perl.exe and perl.exe and renamed perl-static.exe to perl.exe and copied fat (static CRT+static XS) perl.exe to t/perl.exe did I get normal harness running and the "No subtests run" failures went away. Changing blead (no steveh vc 2015 patches) to static CRT with VC 2003 also causes "No subtests run". Are you seeing harness fail like this? Is there something special you are doing?
I hadn't tried BUILD_STATIC + ALL_STATIC. I just applied the v4 patch plus your fixes (which I've merged together into the attached patch\, which is what I intend to apply\, but will wait now until you've got it working too) and did a standard "nmake CCTYPE=MSVC140 && nmake CCTYPE=MSVC140 test" in a VC14 command prompt (set up by running C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat\, with WIN64=undef set too).
For a static CRT test with an old VS (I tried VS2010) I simply changed LIBC from msvcrt.lib to libcmt.lib and -MD to -MT in the Makefile and did a standard "nmake CCTYPE=MSVC100 && nmake CCTYPE=MSVC100 test" in a VC10 command prompt (C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\vcvarsall.bat + WIN64=undef).
I've now tried the VS2010 build with both the static CRT and BUILD_STATIC + ALL_STATIC\, and that works fine too. (It hasn't finished all tests yet\, but is certainly working normally -- it's done all the tests under t/ with only a couple of failures in porting/utils.t and perf/taint.t and is now well into the cpan/ sub-directories' tests.)
Are you sure you're starting from a clean workspace each time? I always do "git clean -dfx" before starting a build to make sure I haven't got any stray files laying around that might interfere with things.
On Thu Aug 06 00:50:03 2015\, Steve.Hay@verosoftware.com wrote:
I hadn't tried BUILD_STATIC + ALL_STATIC. I just applied the v4 patch plus your fixes (which I've merged together into the attached patch\, which is what I intend to apply\, but will wait now until you've got it working too) and did a standard "nmake CCTYPE=MSVC140 && nmake CCTYPE=MSVC140 test" in a VC14 command prompt (set up by running C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat\, with WIN64=undef set too).
For a static CRT test with an old VS (I tried VS2010) I simply changed LIBC from msvcrt.lib to libcmt.lib and -MD to -MT in the Makefile and did a standard "nmake CCTYPE=MSVC100 && nmake CCTYPE=MSVC100 test" in a VC10 command prompt (C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\vcvarsall.bat + WIN64=undef).
I've now tried the VS2010 build with both the static CRT and BUILD_STATIC + ALL_STATIC\, and that works fine too. (It hasn't finished all tests yet\, but is certainly working normally -- it's done all the tests under t/ with only a couple of failures in porting/utils.t and perf/taint.t and is now well into the cpan/ sub- directories' tests.)
Are you sure you're starting from a clean workspace each time? I always do "git clean -dfx" before starting a build to make sure I haven't got any stray files laying around that might interfere with things.
I tried blead with VC 2008 on 2K3 with
win32/Makefile | 4 ++-- 1 file changed\, 2 insertions(+)\, 2 deletions(-)
And got harness breakage, but a little bit different from 2003 on XP and 2015 on Win 7.
C:\p523\src\t> perl.exe harness
base/cond.t ....................................................... No subtests
run
base/if.t ......................................................... 1..2
ok 1
ok 2
base/if.t ......................................................... No subtests
run
base/lex.t ........................................................ 1..103
#1 :x: eq :x:
ok 1
ok 2
ok 3
ok 4
ok 5
ok 6
ok 7
ok 8
ok 9
ok 10
ok 11
ok 12 - make sure single quotes are honored \nnot ok
ok 13
ok 14
ok 15
ok 16
ok 17
ok 18 - was the test for the deprecated use of bare \<\< to mean \<\<""
ok 19
ok 20
ok 21
ok 22
ok 23
ok 24
ok 25
ok 26
ok 27
ok 28
ok 29
ok 30
ok 31
ok 32
ok 33
ok 34
ok 35
ok 36
ok 37
ok 38
ok 39
# uggedaboudit
ok 40
ok 41
ok 42
ok 43
ok 44
ok 45
ok 46
ok 47
ok 48
ok 49
ok 50
ok 51
ok 52
ok 53
ok 54
ok 55 - heredoc after "" in s/// in eval
ok 56 - heredoc in "" in multiline s///e in eval
ok 57 - null on same line as heredoc in s/// in eval
ok 58 - heredoc in "" in single-line s///e in eval
ok 59 - heredoc in "" in multiline s///e outside eval
ok 60 - s/// in s/// pattern
ok 61 - here-doc in re-eval
ok 62 - here-doc in re-eval in string eval
ok 63 - eval ending with semicolon
ok 64 - here-doc in single-line re-eval
ok 65 - here-doc in quotes in multiline re-eval
ok 66 - eval 's//\<\<END/' does not leave extra newlines
ok 67 - # after null in s/// repl
ok 68 - s//'#' . \<\<END/e
ok 69 - s//3}->{3/e
ok 70 - s//${\%x}{3}/e
ok 71 - s/${foo#}//e
ok 72 - listop({ok 70 => 1} + 1)
ok 73 - [perl #105924] require WORD \<\< ...
ok 74 - [perl #105924] goto WORD \<\< ...
ok 75 - [perl #105924] last WORD \<\< ...
ok 76 - [perl #105924] next WORD \<\< ...
ok 77 - [perl #105924] redo WORD \<\< ...
ok 78 - [perl #105924] dump WORD \<\< ...
ok 79 - Use v[0-9]+ as a label
ok 80 - Use v[0-9]+ as a label with space before colon
ok 81 - call a function in package v10::foo
ok 82 - colon detection after vstring does not break ? vstring :
#not
ok 83 - print vstring prints the vstring
ok 84 - q \
C:\p523\src\t>1..2 ok 1 ok 2
C:\p523\src\t>
My VC 2003 perl-static.exe has __app_type 1 (aka _CONSOLE_APP) encoded as a constant in .data section (no __set_app_type calls)\, my VC 2003 static XS+static CRT perl523.dll has __app_type 0 (aka _UNKNOWN_APP) (no __set_app_type calls). msvcr71.dll exports a function called __set_app_type that changes this. Regular DLL CRT perl.exe calls __set_app_type(1) from mainCRTStartup from perl.exe to msvcr71.dll. I need to do more research on how __app_type is selected\, how that data symbol knows how to get initialized\, does the VC linker special case __app_type from -subsystem https://msdn.microsoft.com/en-us/library/fcc1zstk.aspx ? IDK. Wild guess\, do you have CONSOLE or _CONSOLE set as an env var when you are building?
-- bulk88 ~ bulk88 at hotmail.com
On Thu Aug 06 17:08:11 2015\, bulk88 wrote:
My VC 2003 perl-static.exe has __app_type 1 (aka _CONSOLE_APP) encoded as a constant in .data section (no __set_app_type calls)\, my VC 2003 static XS+static CRT perl523.dll has __app_type 0 (aka _UNKNOWN_APP) (no __set_app_type calls). msvcr71.dll exports a function called __set_app_type that changes this. Regular DLL CRT perl.exe calls __set_app_type(1) from mainCRTStartup from perl.exe to msvcr71.dll. I need to do more research on how __app_type is selected\, how that data symbol knows how to get initialized\, does the VC linker special case __app_type from -subsystem https://msdn.microsoft.com/en- us/library/fcc1zstk.aspx ? IDK. Wild guess\, do you have CONSOLE or _CONSOLE set as an env var when you are building?
-subsystem:console\,"5.01" on every call to link.exe didn't fix the harness problem or change anything about it.
I solved the harness with perl523.dll with static CRT sees no child stdout data problem with this crude patch\, the patch works for VC 2003 and VC 2008.
The
+#define _UNKNOWN_APP 0 +#define _CONSOLE_APP 1 +#define _GUI_APP 2 + +EXTERN_C _CRTIMP void __cdecl __set_app_type(int);
stuff should probably be moved to the start of perllib.c or to win32.h inside PERL_CORE.
win32/Makefile | 4 ++-- win32/perllib.c | 6 ++++++ 2 files changed\, 8 insertions(+)\, 2 deletions(-)
How come you didn't have to do this?
-- bulk88 ~ bulk88 at hotmail.com
On Thu Aug 06 20:19:40 2015\, bulk88 wrote:
I solved the harness with perl523.dll with static CRT sees no child stdout data problem with this crude patch\, the patch works for VC 2003 and VC 2008.
VC 2010 also fails for me with static CRT+harness not seeing any output from from child procs.
-- bulk88 ~ bulk88 at hotmail.com
On 7 August 2015 at 01:08\, bulk88 via RT \perlbug\-followup@​perl\.org wrote:
On Thu Aug 06 00:50:03 2015\, Steve.Hay@verosoftware.com wrote:
I hadn't tried BUILD_STATIC + ALL_STATIC. I just applied the v4 patch plus your fixes (which I've merged together into the attached patch\, which is what I intend to apply\, but will wait now until you've got it working too) and did a standard "nmake CCTYPE=MSVC140 && nmake CCTYPE=MSVC140 test" in a VC14 command prompt (set up by running C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat\, with WIN64=undef set too).
For a static CRT test with an old VS (I tried VS2010) I simply changed LIBC from msvcrt.lib to libcmt.lib and -MD to -MT in the Makefile and did a standard "nmake CCTYPE=MSVC100 && nmake CCTYPE=MSVC100 test" in a VC10 command prompt (C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\vcvarsall.bat + WIN64=undef).
I've now tried the VS2010 build with both the static CRT and BUILD_STATIC + ALL_STATIC\, and that works fine too. (It hasn't finished all tests yet\, but is certainly working normally -- it's done all the tests under t/ with only a couple of failures in porting/utils.t and perf/taint.t and is now well into the cpan/ sub- directories' tests.)
Are you sure you're starting from a clean workspace each time? I always do "git clean -dfx" before starting a build to make sure I haven't got any stray files laying around that might interfere with things.
I tried blead with VC 2008 on 2K3 with ----------------------------------------------------------------------- win32/Makefile | 4 ++-- 1 file changed\, 2 insertions(+)\, 2 deletions(-)
diff --git a/win32/Makefile b/win32/Makefile index 23adcd4..597e894 100644 --- a/win32/Makefile +++ b/win32/Makefile @@ -443\,7 +443\,7 @@ DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT LOCDEFS = -DPERLDLL -DPERL_CORE CXX_FLAG = -TP -EHsc
-LIBC = msvcrt.lib +LIBC = libcmt.lib
!IF "$(CFG)" == "Debug" OPTIMIZE = -Od -MD -Zi -DDEBUGGING @@ -459\,7 +459\,7 @@ OPTIMIZE = -Od -MDd -Zi -D_DEBUG -DDEBUGGING LINK_DBG = -debug !ELSE # -O1 yields smaller code\, which turns out to be faster than -O2 on x86 and x64 -OPTIMIZE = -O1 -MD -Zi -DNDEBUG -GS- +OPTIMIZE = -Od -LD -Zi -DNDEBUG -GS- # we enable debug symbols in release builds also LINK_DBG = -debug -opt:ref\,icf # you may want to enable this if you want COFF symbols in the executables -----------------------------------------------------------------------
I tried -MT rather than -LD\, but I've just tried -LD too and it makes no difference -- mine still works fine.
I notice that your diff above includes -GS- in the OPTIMIZE flags. That isn't in blead. Where has that come from? Do you have other patches in place too? Does a clean blead work with just the switch to libcmt.lib / -MT and nothing else (i.e. the attached patch)?
My VC 2003 perl-static.exe has __app_type 1 (aka _CONSOLE_APP) encoded as a constant in .data section (no __set_app_type calls)\, my VC 2003 static XS+static CRT perl523.dll has __app_type 0 (aka _UNKNOWN_APP) (no __set_app_type calls). msvcr71.dll exports a function called __set_app_type that changes this. Regular DLL CRT perl.exe calls __set_app_type(1) from mainCRTStartup from perl.exe to msvcr71.dll. I need to do more research on how __app_type is selected\, how that data symbol knows how to get initialized\, does the VC linker special case __app_type from -subsystem https://msdn.microsoft.com/en-us/library/fcc1zstk.aspx ? IDK. Wild guess\, do you have CONSOLE or _CONSOLE set as an env var when you are building?
No\, I don't have either CONSOLE or _Console set in the environment.
On Fri Aug 07 01:01:08 2015\, Steve.Hay@verosoftware.com wrote:
I tried -MT rather than -LD\, but I've just tried -LD too and it makes no difference -- mine still works fine.
I notice that your diff above includes -GS- in the OPTIMIZE flags. That isn't in blead. Where has that come from? Do you have other patches in place too? Does a clean blead work with just the switch to libcmt.lib / -MT and nothing else (i.e. the attached patch)?
Trying a clean blead with VC 2003 on WinXP.
../cpan/JSON-PP/t/016_tied.t ...................................... Warning: una ble to close filehandle properly: Bad file descriptor during global destruction.
../cpan/JSON-PP/t/016_tied.t ...................................... No subtests run ../cpan/JSON-PP/t/017_relaxed.t ................................... Warning: una ble to close filehandle properly: Bad file descriptor during global destruction.
../cpan/JSON-PP/t/017_relaxed.t ................................... No subtests run ../cpan/JSON-PP/t/018_json_checker.t .............................. Warning: una ble to close filehandle properly: Bad file descriptor during global destruction.
../cpan/JSON-PP/t/018_json_checker.t .............................. No subtests run ../cpan/JSON-PP/t/019_incr.t ...................................... Warning: una ble to close filehandle properly: Bad file descriptor during global destruction.
Terminating on signal SIGINT(2) Terminating on signal SIGINT(2) NMAKE : fatal error U1058: terminated by user Stop.
C:\perl521\src\win32> C:\perl521\src\win32>git log -n3 WARNING: terminal is not fully functional commit 57da2ba98354a7c54f3141a6e5541fd50186d376 Author: Daniel Dragan \bulk88@​hotmail\.com Date: Fri Aug 7 06:02:44 2015 -0400
static crt form shay
commit 749f0eb1485a7bb7561d71f9539d9b7655363136 Author: Steve Hay \steve\.m\.hay@​googlemail\.com Date: Fri Aug 7 08:39:07 2015 +0100
CPAN-Meta-YAML is now synced with 0.017-TRIAL
Delete Changes file. We don't put IGNORABLEs in cpan/ folders.
commit 5356d32ec564e8c2e19758f66dbcbbc38327aa75 Author: Tony Cook \tony@​develop\-help\.com Date: Fri Aug 7 14:20:36 2015 +1000
remove the byte-order-mark introduced to sv.c by 5488d373
This causes the build to fail on NetBSD
C:\perl521\src\win32>nmake test CCTYPE=MSVC70
No\, I don't have either CONSOLE or _Console set in the environment.
Email/offlist mail me your VC 2013 or VC 2010 (I'd prefer VC 2010)\, static CRT perl.exe\, perl.pdb\, perl523.dll and perl523.pdb files. I want to know what app_type is initially set with and where it is being changed inside you particular binaries. Push your static CRT branch somewhere public. I think problem needs to be solved by a 3rd person trying to compile static CRT perl. That 3rd person will probably be TonyC unless A. Sinan Unur comes back.
Same checkout as above\, VC 2003 with Win2k (a rarely used system of mine)\, same harness problem
xcopy /f /r /i /d /y ..\perlglob.exe ..\t\ C:\p523\staticcrt\perlglob.exe -> C:\p523\staticcrt\t\perlglob.exe 1 File(s) copied set PERL_STATIC_EXT=Win32CORE cd ..\t perl.exe harness base/cond.t ....................................................... No subtests run base/if.t ......................................................... No subtests run base/lex.t ........................................................ No subtests run base/num.t ........................................................ No subtests run base/pat.t ........................................................ No subtests run base/rs.t ......................................................... No subtests run base/term.t ....................................................... No subtests run base/translate.t .................................................. No subtests
Retried your commit with VC 2013 on Win 7\, same problem remains. 4 different PCs\, 5 different VCs. Not one of my PCs work for static CRT perl harness. What OS are you using? Are using cmd.exe or powershell to run nmake?
-- bulk88 ~ bulk88 at hotmail.com
Replied offlist with requested files. This is a standard cmd.exe on Windows 8.1. I will try another machine (Win 7) tonight...
-----Original Message----- From: bulk88 via RT [mailto:perlbug-followup@perl.org] Sent: 07 August 2015 13:14 Cc: perl5-porters@perl.org Subject: [perl #125714] Adding support for VS2015 (VC14)
Retried your commit with VC 2013 on Win 7\, same problem remains. 4 different PCs\, 5 different VCs. Not one of my PCs work for static CRT perl harness. What OS are you using? Are using cmd.exe or powershell to run nmake?
On Fri Aug 07 07:16:52 2015\, Steve.Hay@verosoftware.com wrote:
Replied offlist with requested files. This is a standard cmd.exe on Windows 8.1. I will try another machine (Win 7) tonight...
I've just tried with exactly the same setup (blead + static-crt.patch using VS2010 Pro SP1) but now on Windows 7 SP1 rather than Windows 8.1 and I now see the same problem as you -- "No subtests run". Very odd indeed!
On Fri Aug 07 13:33:31 2015\, shay wrote:
I've just tried with exactly the same setup (blead + static-crt.patch using VS2010 Pro SP1) but now on Windows 7 SP1 rather than Windows 8.1 and I now see the same problem as you -- "No subtests run". Very odd indeed!
The binaries you sent me.
perl.exe\, __app_type initialized to 1 (aka _CONSOLE_APP)\, no explicit set funcs called perl523.dll\, __app_type initialized to 0 (aka _UNKNOWN_APP)\, no explicit set funcs called
This 2 values are identical to all static CRT perls I've built myself.
If you grep the CRT source code\, you will see __app_type being used in _free_osfhnd and _set_osfhnd (the unexported function\, not P5's pioinfo macro). __app_type seems to be used\, from what I can tell\, whether fd 0/1/2 are stdout/stdin/stderr (this process has a console)\, or regular disk file or serial ports created with open()/fopen() (this is a GUI process with no console). __app_type controls synchronizing the CRT's view of what OS handles are IN/OUT/ERR with Win32 API/layer/subsystem stdout/stdin/stderr handles (ie\, SetStdHandle()). IDK why it works on Win 8.1 for you\, but fails on every win OS older than 8.1\, I dont have access to 8.1 to give any comments. If the CRT never calls SetStdHandle()\, the gazillion dup2/fdreopen calls that Perl core\, IPC::* and TAP::Harness do to capture child proc's STDOUT wont have any effect across process boundaries since Win32 subsystem was never informed about the change/dup2\, and some parts of Win32 Perl core use GetStdHandle http://perl5.git.perl.org/perl.git/blob/HEAD:/win32/win32.c#l4458
-- bulk88 ~ bulk88 at hotmail.com
There has sadly been no progress on supporting building perl with VS2015\, which is looking really bad now with VS2017 just around the corner.
None of the solutions posted so far on this ticket seem very nice (hence why it has languished for so long). Personally\, I find being able to build with current versions of VC++ more important than the fixes for #120091/#118059 which were applied in commit https://perl5.git.perl.org/perl.git/commit/b47a847f62
That commit reverted parts of https://perl5.git.perl.org/perl.git/commit/9b1f18150a and https://perl5.git.perl.org/perl.git/commit/46e77f1118, thereby reintroducing use of the ioinfo struct\, which is what causes all the trouble with building with VS2015 because of the big CRT shake-up in that version.
Simply reverting commit b47a847f62 (and thus undoing the #120091/#118059 fixes) allows building perl with VS2015 with few other changes required. I've attached a patch against perl-5.24.1 which does exactly this.
I didn't see the dist\IO\t\cachepropagate-tcp.t failure which was the subject of #118059 when I did the build\, though that was only an intermittent failure anyway. However\, the underlying problem never previously caused me any trouble that I'm aware of in day-to-day use so for me this works a treat. (I think it can be a pain for smokers\, though...)
The importance of this for me is that I've found that linking code built with VS2015 against libraries built with VS2013 doesn't work: it causes "unresolved external symbol: ___iob_func" errors. The solution appears to be a need to rebuild the VS2013 libraries with VS2015 (e.g. see http://stackoverflow.com/questions/30412951/unresolved-external-symbol-imp-fprintf-and-imp-iob-func-sdl2 -- in particular\, see MarkH's response to a seemingly simple "fix"\, explaining that it won't actually work).
Thus\, having updated all my other software from VS2013 to VS2015\, I now have to rebuild perl with VS2015 as well otherwise I can't link my other software against perl524.lib. Fixing that problem is far more important than fixing #118059\, hence I personally am going with the attached patch.
The question is how many other people will find themselves in the same position?
It's rather late in the day for 5.26 (with user-visible change freeze already gone\, and full code freeze coming in three weeks) but I'm sorely tempted to apply something like the attached patch to blead so that people can actually build perl with the current VS compiler suite. It looks really bad that with VS2017 almost upon us we still haven't even got support for VS2015 in yet.
If we did this then of course I'd be in support of reverting the patch sometime in the future if/when we get a decent solution for supporting VS2015 with the #120091/#118059 fixes in place as well. But until we can do both things together in a sensible manner I feel that supporting VS2015 is more important than fixing #118059 -- especially since discovering the ___iob_func problem\, which rules out sticking with VS2013 for perl if you want to upgrade other software tha links against it.
How do other people feel about this suggestion\, and is it too late for 5.26?
There has sadly been no progress on supporting building perl with VS2015\, which is looking really bad now with VS2017 just around the corner.
None of the solutions posted so far on this ticket seem very nice (hence why it has languished for so long). Personally\, I find being able to build with current versions of VC++ more important than the fixes for #120091/#118059 which were applied in commit https://perl5.git.perl.org/perl.git/commit/b47a847f62
That commit reverted parts of https://perl5.git.perl.org/perl.git/commit/9b1f18150a and https://perl5.git.perl.org/perl.git/commit/46e77f1118, thereby reintroducing use of the ioinfo struct\, which is what causes all the trouble with building with VS2015 because of the big CRT shake-up in that version.
Simply reverting commit b47a847f62 (and thus undoing the #120091/#118059 fixes) allows building perl with VS2015 with few other changes required. I've attached a patch against perl-5.24.1 which does exactly this.
I didn't see the dist\IO\t\cachepropagate-tcp.t failure which was the subject of #118059 when I did the build\, though that was only an intermittent failure anyway. However\, the underlying problem never previously caused me any trouble that I'm aware of in day-to-day use so for me this works a treat. (I think it can be a pain for smokers\, though...)
The importance of this for me is that I've found that linking code built with VS2015 against libraries built with VS2013 doesn't work: it causes "unresolved external symbol: ___iob_func" errors. The solution appears to be a need to rebuild the VS2013 libraries with VS2015 (e.g. see http://stackoverflow.com/questions/30412951/unresolved-external-symbol-imp-fprintf-and-imp-iob-func-sdl2 -- in particular\, see MarkH's response to a seemingly simple "fix"\, explaining that it won't actually work).
Thus\, having updated all my other software from VS2013 to VS2015\, I now have to rebuild perl with VS2015 as well otherwise I can't link my other software against perl524.lib. Fixing that problem is far more important than fixing #118059\, hence I personally am going with the attached patch.
The question is how many other people will find themselves in the same position?
It's rather late in the day for 5.26 (with user-visible change freeze already gone\, and full code freeze coming in three weeks) but I'm sorely tempted to apply something like the attached patch to blead so that people can actually build perl with the current VS compiler suite. It looks really bad that with VS2017 almost upon us we still haven't even got support for VS2015 in yet.
If we did this then of course I'd be in support of reverting the patch sometime in the future if/when we get a decent solution for supporting VS2015 with the #120091/#118059 fixes in place as well. But until we can do both things together in a sensible manner I feel that supporting VS2015 is more important than fixing #118059 -- especially since discovering the ___iob_func problem\, which rules out sticking with VS2013 for perl if you want to upgrade other software tha links against it.
How do other people feel about this suggestion\, and is it too late for 5.26?
On 01/27/2017 07:25 PM\, Steve Hay via RT wrote:
There has sadly been no progress on supporting building perl with VS2015\, which is looking really bad now with VS2017 just around the corner.
None of the solutions posted so far on this ticket seem very nice (hence why it has languished for so long). Personally\, I find being able to build with current versions of VC++ more important than the fixes for #120091/#118059 which were applied in commit https://perl5.git.perl.org/perl.git/commit/b47a847f62
That commit reverted parts of https://perl5.git.perl.org/perl.git/commit/9b1f18150a and https://perl5.git.perl.org/perl.git/commit/46e77f1118, thereby reintroducing use of the ioinfo struct\, which is what causes all the trouble with building with VS2015 because of the big CRT shake-up in that version.
Simply reverting commit b47a847f62 (and thus undoing the #120091/#118059 fixes) allows building perl with VS2015 with few other changes required. I've attached a patch against perl-5.24.1 which does exactly this.
I didn't see the dist\IO\t\cachepropagate-tcp.t failure which was the subject of #118059 when I did the build\, though that was only an intermittent failure anyway. However\, the underlying problem never previously caused me any trouble that I'm aware of in day-to-day use so for me this works a treat. (I think it can be a pain for smokers\, though...)
The importance of this for me is that I've found that linking code built with VS2015 against libraries built with VS2013 doesn't work: it causes "unresolved external symbol: ___iob_func" errors. The solution appears to be a need to rebuild the VS2013 libraries with VS2015 (e.g. see http://stackoverflow.com/questions/30412951/unresolved-external-symbol-imp-fprintf-and-imp-iob-func-sdl2 -- in particular\, see MarkH's response to a seemingly simple "fix"\, explaining that it won't actually work).
Thus\, having updated all my other software from VS2013 to VS2015\, I now have to rebuild perl with VS2015 as well otherwise I can't link my other software against perl524.lib. Fixing that problem is far more important than fixing #118059\, hence I personally am going with the attached patch.
The question is how many other people will find themselves in the same position?
It's rather late in the day for 5.26 (with user-visible change freeze already gone\, and full code freeze coming in three weeks) but I'm sorely tempted to apply something like the attached patch to blead so that people can actually build perl with the current VS compiler suite. It looks really bad that with VS2017 almost upon us we still haven't even got support for VS2015 in yet.
If we did this then of course I'd be in support of reverting the patch sometime in the future if/when we get a decent solution for supporting VS2015 with the #120091/#118059 fixes in place as well. But until we can do both things together in a sensible manner I feel that supporting VS2015 is more important than fixing #118059 -- especially since discovering the ___iob_func problem\, which rules out sticking with VS2013 for perl if you want to upgrade other software tha links against it.
How do other people feel about this suggestion\, and is it too late for 5.26?
While it's not fun to introduce things in freeze\, 5.25.10 is the full freeze and until then fixes can be considered.
I definitely think we should consider reverting this to allow compiling. A race condition is a problem\, but it's intermittent. Not being able to run on VC2015 is a consistent problem\, and a problematic one at that. :)
Can anyone please weigh in on this?
On 2017/02/07 4:15\, Sawyer X wrote:
On 01/27/2017 07:25 PM\, Steve Hay via RT wrote:
There has sadly been no progress on supporting building perl with VS2015\, which is looking really bad now with VS2017 just around the corner.
None of the solutions posted so far on this ticket seem very nice (hence why it has languished for so long). Personally\, I find being able to build with current versions of VC++ more important than the fixes for #120091/#118059 which were applied in commit https://perl5.git.perl.org/perl.git/commit/b47a847f62
That commit reverted parts of https://perl5.git.perl.org/perl.git/commit/9b1f18150a and https://perl5.git.perl.org/perl.git/commit/46e77f1118, thereby reintroducing use of the ioinfo struct\, which is what causes all the trouble with building with VS2015 because of the big CRT shake-up in that version.
Simply reverting commit b47a847f62 (and thus undoing the #120091/#118059 fixes) allows building perl with VS2015 with few other changes required. I've attached a patch against perl-5.24.1 which does exactly this.
I didn't see the dist\IO\t\cachepropagate-tcp.t failure which was the subject of #118059 when I did the build\, though that was only an intermittent failure anyway. However\, the underlying problem never previously caused me any trouble that I'm aware of in day-to-day use so for me this works a treat. (I think it can be a pain for smokers\, though...)
The importance of this for me is that I've found that linking code built with VS2015 against libraries built with VS2013 doesn't work: it causes "unresolved external symbol: ___iob_func" errors. The solution appears to be a need to rebuild the VS2013 libraries with VS2015 (e.g. see http://stackoverflow.com/questions/30412951/unresolved-external-symbol-imp-fprintf-and-imp-iob-func-sdl2 -- in particular\, see MarkH's response to a seemingly simple "fix"\, explaining that it won't actually work).
Thus\, having updated all my other software from VS2013 to VS2015\, I now have to rebuild perl with VS2015 as well otherwise I can't link my other software against perl524.lib. Fixing that problem is far more important than fixing #118059\, hence I personally am going with the attached patch.
The question is how many other people will find themselves in the same position?
It's rather late in the day for 5.26 (with user-visible change freeze already gone\, and full code freeze coming in three weeks) but I'm sorely tempted to apply something like the attached patch to blead so that people can actually build perl with the current VS compiler suite. It looks really bad that with VS2017 almost upon us we still haven't even got support for VS2015 in yet.
If we did this then of course I'd be in support of reverting the patch sometime in the future if/when we get a decent solution for supporting VS2015 with the #120091/#118059 fixes in place as well. But until we can do both things together in a sensible manner I feel that supporting VS2015 is more important than fixing #118059 -- especially since discovering the ___iob_func problem\, which rules out sticking with VS2013 for perl if you want to upgrade other software tha links against it.
How do other people feel about this suggestion\, and is it too late for 5.26?
While it's not fun to introduce things in freeze\, 5.25.10 is the full freeze and until then fixes can be considered.
I definitely think we should consider reverting this to allow compiling. A race condition is a problem\, but it's intermittent. Not being able to run on VC2015 is a consistent problem\, and a problematic one at that. :)
Can anyone please weigh in on this?
I think it's worth to reverting this to build on VS2015. This allow people build perl easily with MS official develop environment virtual machine.
https://developer.microsoft.com/en-us/windows/downloads/virtual-machines
This is now applied to blead by commit 1f664ef5314fb6e438137c44c95cf5ecdbdb5e9b\, modified slightly to only revert the RT#120091/118059 fixes for VS2015 (and higher)\, as per further discussion that I had off-list with the ActivePerl and StrawberryPerl maintainers (emails now attached here for future reference).
Thus\, compilation with GCC and VS up to and including VS2013 should be materially the same as before this patch; compilation with VS2015 is now possible at the price of reintroducing the RT#120091/118059 bugs.
Gmail *Steve Hay \steve\.m\.hay@​googlemail\.com*
*VC++ 2015 support for Perl* 9 messages
*Steve Hay *\steve\.m\.hay@​googlemail\.com 13 February 2017 at 08:24 To: Andy Grundman \andyg@​activestate\.com\, Jan Dubois \jan@​jandubois\.com\, kmx \kmx@​atlas\.cz Hi\,
I realize ActivePerl and StrawberryPerl both use gcc rather than VC++\, but I wondered if any of you had any feelings either way about my proposal to revert a fix for perl#120091/118059 in order to support building with VC++ 2015?
I posted a patch recently here: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=125714#txn-1446150 \https://rt-archive.perl.org/perl5/Ticket/Display.html?id=125714#txn-1446150 Nobody has replied other than Sawyer... asking for more input\, but none has been forthcoming :-/
Am I the only one that cares about Perl with VC++?!
The VC++ aspect might not interest you\, but how would you feel about the bug fix to not close a freed socket OS handle being reverted? I think the main impact it might have is in causing an intermittent test failure in dist/IO/t/cachepropagate-tcp.t\, which might cause some smoker noise (if we even have any Win32 smokers any more).
Thanks\, Steve
*Jan Dubois *\jan@​jandubois\.com 13 February 2017 at 20:06 To: Steve Hay \steve\.m\.hay@​googlemail\.com Cc: Andy Grundman \andyg@​activestate\.com\, kmx \kmx@​atlas\.cz Hi Steve\,
I have a hard time wrapping my head around those patches; and I still feel as uneasy about them as when they were first added.
In general I consider the whole PERL_ITHREADS stuff a failure\, and would consider any bugs that occur only under multiple threads less important than keeping compiler support for VS working. The only caveat is that several CPAN modules use threading for client/server network testing\, so those uses should not be broken.
However\, I think the close socket issue potentially affects all code\, so it might be bad to loose this for mingw compilation without any offsetting benefit. I have never run into it myself though\, so I have no idea how common it is in practice.
What do you think about conditionally reverting the change for VS compilation only? This will of course break binary compatibility between mingw and vs compiled modules\, but I wonder if that is important. It only affects binary package distribution ala PPM\, and if you want to use PPM\, I don't see why you can't use a mingw compiled Perl. If using conditional compilation is a possible solution\, then there should be some indication to mark the VS perl5xx.dll as incompatible with the mingw one.
Sorry for just the high-level feedback right now\, but just looking at the patches again in detail makes my head hurt...
Feel free to quote my reply to p5p or Sawyer\, or let me know if you want me to respond to your thread directly.
Cheers\, -Jan
[Quoted text hidden]
*Jan Dubois *\jan@​jandubois\.com 13 February 2017 at 20:10 To: Steve Hay \steve\.m\.hay@​googlemail\.com Cc: Andy Grundman \andyg@​activestate\.com\, kmx \kmx@​atlas\.cz
On Mon\, Feb 13\, 2017 at 12:06 PM\, Jan Dubois \<jan@jandubois.com \mailto​:jan@​jandubois\.com> wrote:
This will of course break binary compatibility between mingw and vs compiled modules\,
Oops\, forgot to add that not being able to compile with vs2015 at all of course breaks compatibility even harder. :)
Which is why I don't consider VS and mingw producing incompatible modules a big deal anymore.
Cheers\, -Jan
*Steve Hay *\steve\.m\.hay@​googlemail\.com 14 February 2017 at 08:18 To: Jan Dubois \jan@​jandubois\.com Cc: Andy Grundman \andyg@​activestate\.com\, kmx \kmx@​atlas\.cz\, Sawyer X \xsawyerx@​gmail\.com Hi Jan\,
Thanks for the reply.
I also don't use ithreads for anything much -- except mod_perl\, which requires a threaded perl on Windows because Apache httpd is always threaded. So I wouldn't want to go breaking anything there either\, but the close socket issue never caused me any trouble in mod_perl either as far as I can recall.
However\, the idea of conditional compilation is something that had also crossed my mind\, but with a slightly different angle to what you suggest: I was wondering about making the close socket reversion specific to VS2015 (and above)\, rather than all VS versions.
MinGW and VS2015+ would still be incompatible\, but it would have the minor advantage of keeping binary compatibility between MinGW and VS builds for VS\<=2013. The downside is that then VS\<=2013 and VS>2015 would be incompatible\, but (a) (as you point out) they're currently VERY incompatible anyway since VS>2015 doesn't build\, and (b) I think they're incompatible anyway because of the ___iob_func problem that I mentioned in my recent post / proposal on #125714.
I'm CC'ing Sawyer on this (and quoting your full replies below) because ultimately he has the unenviable task of deciding whether to jump with this\, and if so then in what direction :-)
Sawyer - Please feel free to quote any or all of this on p5p if you wish to make any decisions being taken here public.
Thanks\, Steve
On 13 February 2017 at 20:10\, Jan Dubois \<jan@jandubois.com \mailto​:jan@​jandubois\.com> wrote:
On Mon\, Feb 13\, 2017 at 12:06 PM\, Jan Dubois \<jan@jandubois.com \mailto​:jan@​jandubois\.com> wrote:
This will of course break binary compatibility between mingw and vs compiled modules\,
Oops\, forgot to add that not being able to compile with vs2015 at all of course breaks compatibility even harder. :)
Which is why I don't consider VS and mingw producing incompatible modules a big deal anymore.
Cheers\, -Jan
[Quoted text hidden]
*Jan Dubois *\jan@​jandubois\.com 14 February 2017 at 17:42 To: Steve Hay \steve\.m\.hay@​googlemail\.com Cc: Andy Grundman \andyg@​activestate\.com\, kmx \kmx@​atlas\.cz\, Sawyer X \xsawyerx@​gmail\.com
On Tue\, Feb 14\, 2017 at 12:18 AM\, Steve Hay \<steve.m.hay@googlemail.com \mailto​:steve\.m\.hay@​googlemail\.com> wrote:
However\, the idea of conditional compilation is something that had also crossed my mind\, but with a slightly different angle to what you suggest: I was wondering about making the close socket reversion specific to VS2015 (and above)\, rather than all VS versions.
Yes\, that sounds fine. You may still want to store a setting in Config.pm for this in case anybody ever compiles an XS module with VS2013 for a Perl compiled with VS2015+. (Actually\, I've already forgotten again\, but I think this change might affect module compilation too\, as struct offsets may change. Or am I confused about this?)
Cheers\, -Jan
*kmx *\kmx@​atlas\.cz 14 February 2017 at 20:37 To: Steve Hay \steve\.m\.hay@​googlemail\.com\, Jan Dubois \jan@​jandubois\.com Cc: Andy Grundman \andyg@​activestate\.com\, Sawyer X \xsawyerx@​gmail\.com
Hi Steve\,
from me only a short notice that the VC related changes you are considering can very unlikely affect strawberry perl builds which will stay gcc-based.
I do not even have a request (or something like that) from strawberry perl users for compatibility with VC perl builds.
What I am considering is to switch 64bit gcc from SJLJ to SEH in the 5.26.x series\, but it is a completely different story.
kmx
[Quoted text hidden]
*Jan Dubois *\jan@​jandubois\.com 14 February 2017 at 20:43 To: kmx \kmx@​atlas\.cz Cc: Steve Hay \steve\.m\.hay@​googlemail\.com\, Andy Grundman \andyg@​activestate\.com\, Sawyer X \xsawyerx@​gmail\.com
On Tue\, Feb 14\, 2017 at 12:37 PM\, kmx \<kmx@atlas.cz \mailto​:kmx@​atlas\.cz> wrote:
the VC related changes you are considering can very unlikely affect strawberry perl builds which will stay gcc-based.
They would affect Strawberry Perl\, which is why I would prefer that they are localized via conditional compilation.
What I am considering is to switch 64bit gcc from SJLJ to SEH in the 5.26.x series\, but it is a completely different story.
I would recommend to chat with Andy about this\, in case he wants to switch over ActivePerl as well. That would keep the PPM archives working for Strawberry Perl\, and would maybe also keep the PDK working with it as well. I don't know however\, if this is even still working anymore\, or if this is still a goal.
Cheers\, -Jan
*Steve Hay *\steve\.m\.hay@​googlemail\.com 16 February 2017 at 08:20 To: Jan Dubois \jan@​jandubois\.com Cc: Andy Grundman \andyg@​activestate\.com\, kmx \kmx@​atlas\.cz\, Sawyer X \xsawyerx@​gmail\.com On 14 February 2017 at 17:42\, Jan Dubois \<jan@jandubois.com \mailto​:jan@​jandubois\.com> wrote:
On Tue\, Feb 14\, 2017 at 12:18 AM\, Steve Hay \<steve.m.hay@googlemail.com \mailto​:steve\.m\.hay@​googlemail\.com> wrote:
However\, the idea of conditional compilation is something that had also crossed my mind\, but with a slightly different angle to what you suggest: I was wondering about making the close socket reversion specific to VS2015 (and above)\, rather than all VS versions.
Yes\, that sounds fine. You may still want to store a setting in Config.pm for this in case anybody ever compiles an XS module with VS2013 for a Perl compiled with VS2015+.
I'm not sure if that's really necessary (and getting new settings into Config can be a pain unless we just do some localized Windows-only thing): It's simple enough just to compare the cc version in perl against the cc version being used to compile the module and complain if one is \<2015 and one is >=2015.
(Actually\, I've already forgotten again\, but I think this change might affect module compilation too\, as struct offsets may change. Or am I confused about this?)
The offsets in the struct handled by the PERLIO_FILE_*() macros will be different in \<2015 compared to >=2015\, but they will be incompatible anyway. For \<2015 there should be no material change once I make the patch specific to >= 2015.
*Jan Dubois *\jan@​jandubois\.com 16 February 2017 at 18:32 To: Steve Hay \steve\.m\.hay@​googlemail\.com Cc: Andy Grundman \andyg@​activestate\.com\, kmx \kmx@​atlas\.cz\, Sawyer X \xsawyerx@​gmail\.com
On Thu\, Feb 16\, 2017 at 12:20 AM\, Steve Hay \<steve.m.hay@googlemail.com \mailto​:steve\.m\.hay@​googlemail\.com> wrote:
> Yes\, that sounds fine. You may still want to store a setting in Config.pm > for this in case anybody ever compiles an XS module with VS2013 for a Perl > compiled with VS2015+.
I'm not sure if that's really necessary (and getting new settings into Config can be a pain unless we just do some localized Windows-only thing): It's simple enough just to compare the cc version in perl against the cc version being used to compile the module and complain if one is \<2015 and one is >=2015.
Ah yes\, I forgot that we have the ccversion already in Config.pm\, so that is sufficient to detect the mismatch at module build time.
Cheers\, -Jan
This is now applied to blead by commit 1f664ef5314fb6e438137c44c95cf5ecdbdb5e9b\, modified slightly to only revert the RT#120091/118059 fixes for VS2015 (and higher)\, as per further discussion that I had off-list with the ActivePerl and StrawberryPerl maintainers (emails now attached here for future reference).
Thus\, compilation with GCC and VS up to and including VS2013 should be materially the same as before this patch; compilation with VS2015 is now possible at the price of reintroducing the RT#120091/118059 bugs.
@steve-m-hay - Status changed from 'open' to 'resolved'
On 02/19/2017 08:40 AM\, Steve Hay via RT wrote:
This is now applied to blead by commit 1f664ef5314fb6e438137c44c95cf5ecdbdb5e9b\,
shay++
On 02/19/2017 05:04 PM\, Karl Williamson wrote:
On 02/19/2017 08:40 AM\, Steve Hay via RT wrote:
This is now applied to blead by commit 1f664ef5314fb6e438137c44c95cf5ecdbdb5e9b\,
shay++
shay++ indeed.
(This issue involved additional discussions off the list to clarify some finer points. Thanks\, Steve\, for taking this on.)
Migrated from rt.perl.org#125714 (status was 'resolved')
Searchable as RT125714$