problem building with msvc

[revelator]

the C compiler fails to build with msvc 2015 and 2017 "error: failed to open message resource file" msvc 2013 builds that part but fails further down the line with ICU because it does not have char16_t. more errors crop up further down the line with msvc 2013 also.

Would be nice with a more descriptive error, but atleast i found out that the step that fails is creating code386.gh.

[jmalak]

I am using msvc 2015 + UCRT 10.0.15063.0 without any similar problem. c:\dev\ow2-build>cl Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24215.1 for x64 Copyright (C) Microsoft Corporation. All rights reserved. usage: cl [ option... ] filename... [ /link linkoption... ] c:\dev\ow2-build>

Below is definitions and notes from ICU header wipfc/icu/common/unicode/platform.h related to char16_t type.

/**
 * \def U_HAVE_CHAR16_T
 * Defines whether the char16_t type is available for UTF-16
 * and u"abc" UTF-16 string literals are supported.
 * This is a new standard type and standard string literal syntax in C++0x
 * but has been available in some compilers before.
 * @internal
 */
#ifdef U_HAVE_CHAR16_T
    /* Use the predefined value. */
#else
    /*
     * Notes:
     * Visual Studio 2010 (_MSC_VER==1600) defines char16_t as a typedef
     * and does not support u"abc" string literals.
     * Visual Studio 2015 (_MSC_VER>=1900) and above adds support for
     * both char16_t and u"abc" string literals.
     * gcc 4.4 defines the __CHAR16_TYPE__ macro to a usable type but
     * does not support u"abc" string literals.
     * C++11 and C11 require support for UTF-16 literals
     * TODO: Fix for plain C. Doesn't work on Mac.
     */
#   if U_CPLUSPLUS_VERSION >= 11 || (defined(__STDC_VERSION__) && __STDC_VERSION__ >= 201112L)
#       define U_HAVE_CHAR16_T 1
#   else
#       define U_HAVE_CHAR16_T 0
#   endif
#endif

/**
 * \def U_CHAR16_IS_TYPEDEF
 * If 1, then char16_t is a typedef and not a real type (yet)
 * @internal
 */
#if (U_PLATFORM == U_PF_AIX) && defined(__cplusplus) &&(U_CPLUSPLUS_VERSION < 11)
// for AIX, uchar.h needs to be included
# include <uchar.h>
# define U_CHAR16_IS_TYPEDEF 1
#elif defined(_MSC_VER) && (_MSC_VER < 1900)
// Versions of Visual Studio/MSVC below 2015 do not support char16_t as a real type,
// and instead use a typedef.  https://msdn.microsoft.com/library/bb531344.aspx
# define U_CHAR16_IS_TYPEDEF 1
#elif defined(__WATCOMC__)
# define U_CHAR16_IS_TYPEDEF 1
#else
# define U_CHAR16_IS_TYPEDEF 0
#endif

Is correct MSVC version reported by OW setvars script? Did you do full clean before new build by clean script?

[revelator]

msvc 2017 here but i also tried with msvc 2015 and 2013 only 2013 can build it if i remove the reference to libucrt.lib from the trap sources. Theres a bug in the icu sources U_CHAR16_IS_TYPEDEF is correct but it fails on setting char16_t to uint16_t so i made a workaround.

#if (U_PLATFORM == U_PF_AIX) && defined(__cplusplus) &&(U_CPLUSPLUS_VERSION < 11)
// for AIX, uchar.h needs to be included
# include <uchar.h>
# define U_CHAR16_IS_TYPEDEF 1
#elif defined(_MSC_VER) && (_MSC_VER < 1900)
// Versions of Visual Studio/MSVC below 2015 do not support char16_t as a real type,
// and instead use a typedef.  https://msdn.microsoft.com/library/bb531344.aspx
#include <cstdint>
typedef uint16_t char16_t; //C2628  
# define U_CHAR16_IS_TYPEDEF 1
#elif defined(__WATCOMC__)
# define U_CHAR16_IS_TYPEDEF 1
#else
# define U_CHAR16_IS_TYPEDEF 0
#endif

in umachine.h since msvc2013 do'es not have that typedef as mentioned int the comment.

msvc version is reported correctly.

And yes did full clean also tried on another machine with freshly downloaded sources and its the same problem :/.

It does not seem to have anything to do with what API i use rather it fails to parse this segment in particular "code$(target_cpu).gh : ./mkcode.exe xcode$(target_cpu).tmp" in master.mif.

mkcode.exe also does not crash.

[revelator]

ok this is getting weird, i forced the developer console to use 10.0.10586.0 instead of the newer 10.0.17134.0 and it works with msvc 2017 now. Any Api above that causes the same problem as before, while any lower version works. There are no errors printed or logged though so your guess is as good as mine as to what causes this.

my little icu fix also works with msvc 2017 but you still have to replace libucrt.lib with libcmt.lib in the 64 bit trap tcp makefile (any way to set the correct library automatically ?).

also mkinf crashes on creating the installers when built with msvc 2013, doing a full build with msvc 2017 now to check if it also crashes with this.

[jmalak]

I will try to install msvc 2017 and higher UCRT to investigate what is wrong. Anyway problem with trap tcp should be fixed. I am not sure if fix build with msvc 2013 has sense now.

[revelator]

Depends, it builds everything and works fine besides mkinf crashing (which it also does when built with msvc 2017).

Seems later ucrt versions break XP compatibility, i guess that is why it fails ?.

There was a report about the message compiler crashing with a similar problem on the MSDN developer site.

Microsoft stated that this was a known problem and that the flag for XP compatibility would be removed in future versions.

[revelator]

Oh hell, yep that was it, the message compiler no longer supports mof with the later ucrt versions.

Do we really need ucrt when linking the executables to the static runtimes ?, or are there other reasons.

[jmalak]

Sorry what you mean by "mof"?

[revelator]

managed object format, in essence what mc uses to log events on OS prior to vista.

i guess theres a -mof switch to mc in the build chain somewhere for msvc builds, or the compiler runs in XP compatibility where its on by default causing mc to fail when using later ucrt where this behaviour has been disabled.

https://docs.microsoft.com/en-us/windows/desktop/WES/message-compiler--mc-exe-

[jmalak]

Thanks for explanation. It looks like that we need review complete MS toolchain options setting for MS VS with regards to target platform. Probably for 64-bit target setup, Windows 7 target and above has sense. Now setup is general. For 32-bit Windows always OW toolchain is used that compatibility with lower version Windows is not a problem.

[revelator]

That might be a good idea. Seems only 64-bit target is affected on windows with MS compiler chain anyway. Not sure why mkinf crashes, need to investigate that next, but the compiler works fine (tested by building RDOS).

[jmalak]

I did a look on mkinf utility and it looks like some memory corruption. I will analyse this issue and fix code.

[revelator]

sounds good :)

[jmalak]

there were a few memory problems, now it should be fixed by f8cc4b686daedb92dda59793246b6bd3b2266a5e commit

[revelator]

Ok will give it a spin :) thanks

[revelator]

That worked splendidly, thanks.

[revelator]

hmm latest version also builds but oh... my... god... wgml is so slow now, im looking at two days of building the documentation and no end in sight yet. Not sure what changed ?

[jmalak]

I found out bug in latest ucrt, that I included fix for it into CRTL extension code. Now it should compile with latest VS 2017 and latest version of UCRT. Slow build can be realated to DOSBOX parameter setup change that I did during OSX build test. I will remove it to be quicker as before, sorry.

[jmalak]

it is fixed by commit fb10bec4bb70988220a2b040536fad2451b98ddd

[revelator]

no problem, just happy it was not on my end with this build problem :). Sounds good with the ucrt fix :)

[revelator]

i was wondering if long filename support was fixed ?, in case its not theres a developer at vogons who fixed it for dosbox and vdos who might be able to help.

[jmalak]

Using DOS emulator is temporary solution until WGML utility will be host native application. Guys from OW1.9 are working on WGML utility source code. Anyway long path names and paths with spaces are still not fully supported by OW2 tools. Most of them handle it properly but there are still some which doesn't handle it.

[revelator]

could have been misinterpreted i should have known :), i ment in regards to the standard compiler tools. He might be willing to help get the rest of them LFN capable.

[revelator]

btw one small bug, the 64 bit wrc crashes, strangely the static bootstrap version works fine.

[jmalak]

Do you have any details about this bug, on which project? It could be again problem with Microsoft ucrt.

[revelator]

debugging shows it crashing in layer0.c in FILE *fp with a NULL pointer. Strangely this only affects the dll build.

[revelator]

the function is res_seek in this part -> if( hInstance.fp == fp ) { if( where == SEEK_SET ) { return( fseek( fp, amount + WResFileShift, SEEK_SET ) != 0 ); // triggers a crash } return( fseek( fp, amount, where ) != 0 ); }

expression cannot be evaluated. - fp 0x0000000000000000 {_Placeholder=??? } _iobuf *

tools used are msvc 2015 and intel compiler 17

[revelator]

spoke to soon, the 32 bit version also crashes but does not pop up the debug window like the 64 bit version does.

[jmalak]

If fp pointer is NULL it looks like image file was not open for reading resource text from image file. For DLL image name is DLL file name and for non-DLL version it is executable (EXE) file. Image file handling function is InitRcMsgs in rcldstr.c. DLL image name is get by get_dllname function in clibext.obj (bld/watcom/binbuild/..) But if image file is not open then you should get error message.

[jmalak]

on which project wrc crash?

[jmalak]

Sorry, I overlook that you use Intel compiler. it should be a problem that _MSC_VER macro is not defined. the get_dllname should be fixed for Intel compiler because it probably returns "" name.

[jmalak]

anyway bwrc doesn't use resources it uses text array it is switched by macro INCL_MSGTEXT

[revelator]

no project i was just trying to run it to see options. Trying a pure msvc build now.

[revelator]

The thread 'Win64 Thread' (0x1e50) has exited with code 0 (0x0). Microsoft Visual Studio C Runtime Library has detected a fatal error in wrc.exe.

this also happens with a pure msvc build so its not caused by the intel tools.

[revelator]

Hmm one problem was the struct for RcFile, it has the same typedef on both sides leading to memory corruption, deleting one side fixed the variables could not be read error, but then it crashes further down the line instead. The struct RcBuffer also has the same type on both sides but if i remove the topmost typedef it goes back to the same error again urgh. Seems msvc is not a happy camper with this code.

[revelator]

32 bit error ->

'Open Watcom Windows and OS/2 Resource Compiler Version 2.0 beta Dec 14 2018 10:35:52 (32-bit) Copyright (c) 2002-2018 The Open Watcom Contributors. All Rights Reserved. Portions Copyright (c) 1993-2002 Sybase, Inc. All Rights Reserved. Source code is available under the Sybase Open Watcom Public License. See http://www.openwatcom.org/ for details.

The instruction at 0x002cb3cf referenced memory at 0x00000010. The memory could not be read. Exception fielded by 0x004037a0 EAX=0x0018f848 EBX=0x00000004 ECX=0x00000000 EDX=0x00000001 ESI=0x00000000 EDI=0x00330335 EBP=0x00000000 ESP=0x0018f7fc EIP=0x002cb3cf EFL=0x00010206 CS =0x00000023 SS =0x0000002b DS =0x0000002b ES =0x0000002b FS =0x00000053 GS =0x0000002b Stack dump (SS:ESP) 0x00000001 0x00000001 0x006c006c 0x0018f848 0x00000004 0x00330335 0x00000000 0x002b57fc 0x00000000 0x00000000 0x0018f848 0x00000000 0x00330335 0x00000000 0x00000000 0x002d6323 0x00000000 0x00000000 0x002d3f3f 0x00000000 0x002c901a 0x002dce01 0x00000000 0x00000000 0x00000000 0x002cf6b5 0x0000006a 0x002dcee4 0x00000000 0x002cf680 0x00330335 0x002cc669 0x0018f9d4 0x0000006a 0x002cc5cd 0x0018f8e0 0x00000003 0x002d1ba0 0x00330335 0x00000600 0x00000700 0x0018f9d4 0x00000000 0x00000001 0x00409078 0x00000000 0x002cc645 0x00000100 0x0018f9d4 0x002c7eca 0x00000007 0x000000ff 0x002c9173 0x575c3a43 0x6f637461 0x49425c6d 0x5c544e4e 0x2e637277 0x00657865 0x0018f97c 0x00000002 0x751c124c 0x00000000 0x00000000 0x0018fba4 0x00000002 0x00000002 0x0018f934 0x751a3046 0x00000000 0x00000200 0x0018f97c'

[jmalak]

I little bit cleanup file caching code. But I am still not able reproduce your problem.

[revelator]

hmm odd then if it does not crash for you it seems to indicate something is fubar on my end. Might be my antivirus i use F-secure client security, and it does tend to get rather pissy about anything accessing memory in ways it finds suspecious. But it has not warned me about anything so far so ?. Though after microsofts meltdown and specter patches it has acted strange at times so im not sure its reliable anymore.

[jmalak]

I found out some memory leaks, but it should not cause the problem which you describe.

[jmalak]

do you get wrc usage info if you doesn't specify anything on command line?

[revelator]

unfortunatly not, i get that long string of exception info above.

[jmalak]

It is some problem with openning DLL image for reading resource text. It is done during initialization in InitRcMsgs function, could you check this code especialy if get_dllname get proper name and DLL image is correctly open?

[revelator]

Sure :). Im not certain but the debugger seems to think that its a FILE pointer error, atleast thats what it tells me. Expression cannot be evaluated in FILE *fp but that should be normal since im not loading any file just running the executable without any arguments. Also rc.exe works fine, though im not sure it uses the exact same code as wrc.exe its pretty strange non the less.

[jmalak]

If you can not display variabl fp then it is optimized to register and doesn't use variable in memory. Try to compile wrc for debugging without optimization. we don't have rc.exe, it is Microsoft. Next problem can be with DLL version, there are two executables. First wrc.exe (command line driver) and second wrcd.dll (own resource compiler)

[revelator]

yep rc.exe is just a wrapper around ms rc.exe, i had a look at the code for it, but the ms rc.exe must be renamed to rc386.exe so atm it does nothing and would probably fail if used. I had a hunch that optimization might be playing tricks on the debugger, ill try a debug build.

[jmalak]

Oh Sorry, it is big misunderstanding. You mean rc.exe MS compatible wrapper for OW Resource compiler. It is another project in bld/mstools I will check, probably it is broken.

[jmalak]

It is only wrapper for wrcd.dll but with MS syntax

[revelator]

uh no the misunderstanding was on my side, i mistakenly assumed that rc.exe was a wrapper around wrc.exe and not a wrapper for MS rc.exe. The bug so far only seems to affect the dll based wrc.exe, the static bootstrap bwrc.exe works fine, so it might as you say be a problem with it loading the dll correctly.

[jmalak]

OW rc.exe is wrapper for wrcd.dll that simulate MS rc syntax/behaviour

[revelator]

it is ?,

`int main( void ) /**/ { int cpuType;

cpuType = GetCpuTypeStub();
switch( cpuType ) {
  case STUB_FOUND_386:
    SpawnProgStub( "rc386.exe" );
    break;
  case STUB_FOUND_X64:
    SpawnProgStub( "rc386.exe" );
    break;
  case STUB_FOUND_AXP:
    SpawnProgStub( "rcaxp.exe" );
    break;
  case STUB_FOUND_PPC:
    SpawnProgStub( "rcppc.exe" );
    break;
  default:
    FatalError( "Unrecognized CPU type" );
}
return( 0 );

}`

this is from rcstub.c seems to me that it needs the ms rc renamed to rc386 or am i completely loosing my mind now Oo.

[jmalak]

all executables in bld/mstools are wrappers for OW tools to simulate appropriate MS commands