open-watcom / open-watcom-v2

Open Watcom V2.0 - Source code repository, Wiki, Latest Binary build, Archived builds including all installers for download.
Other
971 stars 159 forks source link

Bootstrapping watcom with intels compiler #55

Open revelator opened 10 years ago

revelator commented 10 years ago

Managed to bootstrap open Watcom with intels compiler.

If someone wants to toy with it heres a howto.

go into the build\mif directory and open the local.mif file with notepad++, scroll down to the bottom untill you reach this

!if !defined( "WATCOM" ) || "$(bld_cpu)" == "x64"

common settings

now change

common release/debug flags

common_flags_rel = -Ox -Zi common_flags_dbg = -Zi !ifeq release 1 common_flags = $(common_flags_rel) !else common_flags = $(common_flags_dbg) !endif

to

common release/debug flags

common_flags_rel = -Ox -Zi -Qtbb common_flags_dbg = -Zi -Qtbb !ifeq release 1 common_flags = $(common_flags_rel) !else common_flags = $(common_flags_dbg) !endif

the -Qtbb flag is for linking with intels threaded buliding blocks (just an example).

a bit further down change

bld settings

bcc = cl -nologo -c bcxx = cl -nologo -c bcl = cl -nologo

blib = lib -nologo

to

bld settings

bcc = icl -nologo -c bcxx = icl -nologo -c bcl = icl -nologo

blib = xilib -nologo

and further down change

standard settings

cc = cl -nologo -c cxx = cl -nologo -c cl = cl -nologo

librarian = lib -nologo

to

standard settings

cc = icl -nologo -c cxx = icl -nologo -c cl = icl -nologo

librarian = xilib -nologo

save and now open intels command shell instead of msvc's cd to the build root and do as you used to with msvc but it now uses intels compiler instead.

Intels compiler is pretty strict compared to msvc so its nice for finding bugs, besides that it also has way better optmization so toy a bit with its switches.

revelator commented 10 years ago

https://sourceforge.net/projects/cbadvanced/files/Open%20Watcom/intel%20cross%20build/

Intel build for testing.

jmalak commented 10 years ago

Please, do you know what C compiler macro identify Intel C compiler? I can include it into wmake bootstrap build and then switch makefiles to Intel compiler configuration similary as for OW and non-OW compilers

revelator commented 10 years ago

From the top of my head heres the ones i know.

Identification __INTEL_COMPILER
Identification ICC Obsolete Identification ECC Obsolete Identification ICL
Version
INTEL_COMPILER VRP
V = Version R = Revision P = Patch

Version __INTEL_COMPILER_BUILD_DATE YYYYMMDD
YYYY = Year MM = Month DD = Day

jmalak commented 10 years ago

I added Intel compiler changes to git repository. Please, check if it works as expected. For testing is necessary to use clean and build script to rebuild bootstrap version of wmake in build/bin subdirectory

revelator commented 10 years ago

Hmm seems to still use msvc's cl instead of intels icl.

revelator commented 10 years ago

http://nadeausoftware.com/articles/2012/10/c_c_tip_how_detect_compiler_name_and_version_using_compiler_predefined_macros

maybe something usefull here.

jmalak commented 10 years ago

Sorry, it was first step. Now you need to modify build.bat to call make utility available with Intel tollchain and appropriate make files in bld/wmake sub-directory. If you can not use any existing then you must create new one which will be working with Intel make utility. This ensure that new bootstrap wmake utility will be compiled by Intel toolchain. As soon as new wmake is available then standard build should use Intel configuration.

revelator commented 10 years ago

Intels compiler uses msvc nmake which seems to be the reason that it tries to use msvc instead. Atm im trying different approaches ill post if i find a solution.

Den 27-05-2014 14:56, Jiří Malák skrev:

Sorry, it was first step. Now you need to modify build.bat to call make utility available with Intel tollchain and appropriate make files in bld/wmake sub-directory. If you can not use any existing then you must create new one which will be working with Intel make utility. This ensure that new bootstrap wmake utility will be compiled by Intel toolchain. As soon as new wmake is available then standard build should use Intel configuration.

— Reply to this email directly or view it on GitHub https://github.com/open-watcom/open-watcom-v2/issues/55#issuecomment-44271535.

jmalak commented 10 years ago

OK. I understand your approach. But right solution for current build process is to create new makefile for bootstrap wmake by Intel compiler. You can simply create by example "intel" make file as copy of existing "nmake" and change compilers executable to Intel and change compilers options too.

revelator commented 10 years ago

Im not sure we are in disagreement here :)

I made some changes and it kinda works besides not being able to forward the INTEL macro to the bootstrap. i still have to define that manually in local.mif or it will default to msvc's cl.exe. funny thing though when wmake takes over it works Oo. I double checked by having it link dynamically to intels dynamic optimization libraries and checking with depends and it lists the intel dll as a dependency.

changes to mglob.c here

const char FAR* MSBuiltIn = { "AS=ml\n" "BC=bc\n"

if (defined( __INTEL_COMPILER ) && defined( _MSC_VER ))

 "CC=icl\n"

elif (defined( __ICC ) && defined( _MSC_VER ))

 "CC=icl\n"

elif (defined( __ICL ) && defined( _MSC_VER ))

 "CC=icl\n"

elif (defined( __ECC ) && defined( _MSC_VER ))

 "CC=icl\n"

else

 "CC=cl\n"

endif

 "COBOL=cobol\n"

if (defined( __INTEL_COMPILER ) && defined( _MSC_VER ))

 "CPP=icl\n"
 "CXX=icl\n"
 "LIB=xilib\n"
 "FOR=ifort\n"

elif (defined( __ICC ) && defined( _MSC_VER ))

 "CPP=icl\n"
 "CXX=icl\n"
 "LIB=xilib\n"
 "FOR=ifort\n"

elif (defined( __ICL ) && defined( _MSC_VER ))

 "CPP=icl\n"
 "CXX=icl\n"
 "LIB=xilib\n"
 "FOR=ifort\n"

elif (defined( __ECC ) && defined( _MSC_VER ))

 "CPP=icl\n"
 "CXX=icl\n"
 "LIB=xilib\n"
 "FOR=ifort\n"

else

 "CPP=cl\n"
 "CXX=cl\n"
 "LIB=lib\n"

endif

 "FOR=fl\n"
 "PASCAL=pl\n"
 "RC=rc\n"
 ".asm.exe:\n"
 "    $(AS) $(AFLAGS) $_.asm\n"
 ".asm.obj:\n"
 "    $(AS) $(AFLAGS) /c $_.asm\n"
 ".c.exe:\n"
 "    $(CC) $(CFLAGS) $_.c\n"
 ".c.obj:\n"
 "    $(CC) $(CFLAGS) /c $_.c\n"
 ".cpp.exe:\n"
 "    $(CPP) $(CPPFLAGS) $_.cpp\n"
 ".cpp.obj:\n"
 "    $(CPP) $(CPPFLAGS) /c $_.cpp\n"
 ".cxx.exe:\n"
 "    $(CXX) $(CXXFLAGS) $_.cxx\n"
 ".cxx.obj:\n"
 "    $(CXX) $(CXXFLAGS) $_.cxx\n"
 ".bas.obj:\n"
 "    $(BC) $(BFLAGS) $_.bas\n"
 ".cbl.exe:\n"
 "    $(COBOL) $(COBFLAGS) $_.cbl, $_.exe;\n"
 ".cbl.obj:\n"
 "    $(COBOL) $(COBFLAGS) $_.cbl;\n"
 ".f.exe:\n"
 "    $(FOR) $(FFLAGS) $_.f\n"
 ".f.obj:\n"
 "    $(FOR) /c $(FFLAGS) $_.f\n"
 ".f90.exe:\n"
 "    $(FOR) $(FFLAGS) $_.f90\n"
 ".f90.obj:\n"
 "    $(FOR) /c $(FFLAGS) $_.f90\n"
 ".for.exe:\n"
 "    $(FOR) $(FFLAGS) $_.for\n"
 ".for.obj:\n"
 "    $(FOR) /c $(FFLAGS) $_.for\n"
 ".pas.exe:\n"
 "    $(PASCAL) $(PFLAGS) $_.pas\n"
 ".pas.obj:\n"
 "    $(PASCAL) /c $(PFLAGS) $_.pas\n"
 ".rc.res:\n"
 "    $(RC) $(RFLAGS) /r $*\n"

};

Den 27-05-2014 21:15, Jiří Malák skrev:

OK. I understand your approach. But right solution for current build process is to create new makefile for bootstrap wmake by Intel compiler. You can simply create by example "intel" make file as copy of existing "nmake" and change compilers executable to Intel and change compilers options too.

— Reply to this email directly or view it on GitHub https://github.com/open-watcom/open-watcom-v2/issues/55#issuecomment-44321783.

jmalak commented 10 years ago

I recommend you to start with build of bootstrap version of wmake by Intel compiler. If you copy bld/wmake/nmake to bld/wmake/intel then add following line on begining of "intel" file CC = icl correct CFLAGS create subdirectory bld/wmake/binbuild from this drectory call "nmake -f ..\intel" it should build bootstrap version of wmake which define INTEL macro and will be used by build process

revelator commented 10 years ago

Tried but still not defining INTEL hmm ?

Shall i upload a copy of the built wmake somewhere ?

Den 27-05-2014 23:14, Jiří Malák skrev:

I recommend you to start with build of bootstrap version of wmake by Intel compiler. If you copy bld/wmake/nmake to bld/wmake/intel then add following line on begining of "intel" file CC = icl correct CFLAGS create subdirectory bld/wmake/binbuild from this drectory call "nmake -f ..\intel" it should build bootstrap version of wmake which define INTEL macro and will be used by build process

— Reply to this email directly or view it on GitHub https://github.com/open-watcom/open-watcom-v2/issues/55#issuecomment-44336313.

jmalak commented 10 years ago

I a liitle changed wmake precompiler if guard that now you can build bootstrap wmake for Intel toolchain by any toolchain by explicitly define INTEL macro. add -D__INTEL_COMPILER to CFLAGS in make file Ensure that build/bin/wmake is first executable on the PATH You can check wmake macros by "wmake -p" or by simple make file !ifdef INTEL !error INTEL is defined !else !error INTEL is undefined !endif

revelator commented 10 years ago

Works now.

It iss very messy though :S

Den 27-05-2014 23:53, Jiří Malák skrev:

I a liitle change wmake precompiler if guard that now you can build wmake by any toolchain and explicitly define INTEL macro. add -D__INTEL_COMPILER to CFLAGS in intel make file It should ensure that INTEL macro will be defined Ensure that build/bin/wmake is first executable on the PATH You can check wmake macros by simple make file !ifdef INTEL !error INTEL is defined !else !error INTEL is undefined !endif

— Reply to this email directly or view it on GitHub https://github.com/open-watcom/open-watcom-v2/issues/55#issuecomment-44340523.

jmalak commented 10 years ago

Yes, it is a little confusing. You need bootstraped wmake which is compiled only by used toolchain tools and define macro about used toolchain for build process. The wmake and builder tools must be prebuild before bootstrap start, because it is control by these two tools. But builder is only internal tool that version colision can not happen. The build system enable OW full build for unsupported hosts (by example 64-bit Windows and 64-bit Linux now) by native compilers and I didn't found better way how to do it then define auxiliary wmake macro for toolchain.

revelator commented 10 years ago

Probably the same thing those that tried using ICC with mingw encountered, seems its near impossible without a make utility that predefines those macros.

jmalak commented 9 years ago

I did change to build makefiles that now support toolchains macros and toolchain is defined by OWTOOLS environment variable. Build system now properly handle Intell toolchain. But what is not tested is first bootstrap build of wmake by nmake. I think it will require to create Intel specific make file intel in bld/wmake and change build script for Intel toolchain.

Please do you have some time to check/fix this Intell stuff in OW build system after introducing toolchain handling in OW build system?

Thanks

revelator commented 9 years ago

Bootstrapping seems to work allright nothing to remark there. Got a problem with bld\idedemo though not sure is a recent change but the project.mk files have a wrong path in them H:\ while i do my build from F:\ ? is this file a generated file, if so it would be a leftover from when i reinstalled my PC so i sure hope so.

jmalak commented 9 years ago

Thanks for quick test and note. It could be a problem with mistake in cleanup processing. These files are generated by ide2make utility and processed by sed utility. Please, could you send me build log for bld\idedemo project.

Thanks

revelator commented 9 years ago

Was a botched clean indeed seems some files where left from a previous build but a second clean got rid of them. Win 7 sometimes locks files it thinks are in use which is a huge pain at times.

Something you could change would be to replace the -Qtbb for the intel build with -fast which tells the intel compiler to use procedural optimizations on the bootstrapped binaries, it does speed up compile times quite a bit and tbb seems to newer be used anyway (actually i think you need to write specific code for it to work since its a thread library).

jmalak commented 9 years ago

I removed -Qtbb option and added new -fast option

revelator commented 9 years ago

Tested with the new changes and all seems to work except the F77 wfc and wtl build which cannot find optflags.h and a few other headers, i tried adjusting the includes in master.mif but either im doing it wrong or something else is interfering with the include paths so i had to add the paths in the source files themself, after that i could do a full compile. Full clean does not fix it unfortunatly so im a bit clueless as to what can cause this.

jmalak commented 9 years ago

Each compiler use a little specific search path list. I did review/correction for WATCOM, GCC and CLANG in the path but not for Intel compiler. I will check list of included directories for Fortran project and correct them to explicitly include each used path.

jmalak commented 9 years ago

I checked include path and all looks OK, nothing problematic. Please could you send me build log created by command "builder buil OWVERBOSE=1" run in bld/f77/wfc and bld/f77/wfl.

revelator commented 9 years ago

Ill do a new build now with OWVERBOSE=1 only just returned home.

revelator commented 9 years ago

============== 09:10:40 F:\open-watcom-v2\bld\f77\wfl\386\ntx64 =============== icl -nologo -c -D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS -W4 -wd4214 -MT -D_STATIC_CPPLIB -Ox -Zi -fast -Foblderr.obj -DNDEBUG -D__NT_X64 -DNT -DFLAT -DWFL__ -I. -I.. -I"../../h" -I"../../../wfc/h" -I"../../../h" -I"F:\open-watcom-v2\bld/comp_cfg/h" -I"F:\open-watcom-v2\bld/watcom/h" ......\c\blderr.c blderr.c ......\c\blderr.c(139): remark #181: argument of type "long" is incompatible with format "%lu", expecting argument of type "unsigned long" sprintf( temp_buff, "%lu", ordered_args[idx].i ); ^

icl -nologo -c -D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS -W4 -wd4214 -MT -D_STATIC_CPPLIB -Ox -Zi -fast -Foerrgrp.obj -DNDEBUG -DNT_X64 -DNT -DFLAT -DWFL__ -I. -I.. -I"../../h" -I"../../../wfc/h" -I"../../../h" -I"F:\open-watcom-v2\bld/comp_cfg/h" -I"F:\open-watcom-v2\bld/watcom/h" ......\wfc\c\errgrp.c errgrp.c icl -nologo -c -D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS -W4 -wd4214 -MT -D_STATIC_CPPLIB -Ox -Zi -fast -Foerrutil.obj -DNDEBUG -DNT_X64 -DNT -DFLAT -DWFL -I. -I.. -I"../../h" -I"../../../wfc/h" -I"../../../h" -I"F:\open-watcom-v2\bld/comp_cfg/h" -I"F:\open-watcom-v2\bld/watcom/h" ......\c\errutil.c errutil.c icl -nologo -c -D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS -W4 -wd4214 -MT -D_STATIC_CPPLIB -Ox -Zi -fast -Foerrrsrc.obj -DNDEBUG -D__NT_X64 -DNT -DFLAT -DWFL -I. -I.. -I"../../h" -I"../../../wfc/h" -I"../../../h" -I"F:\open-watcom-v2\bld/comp_cfg/h" -I"F:\open-watcom-v2\bld/watcom/h" ......\c\errrsrc.c errrsrc.c icl -nologo -c -D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS -W4 -wd4214 -MT -D_STATIC_CPPLIB -Ox -Zi -fast -Foerrrtns.obj -DNDEBUG -DNT_X64 -DNT -DFLAT -DWFL__ -I. -I.. -I"../../h" -I"../../../wfc/h" -I"../../../h" -I"F:\open-watcom-v2\bld/comp_cfg/h" -I"F:\open-watcom-v2\bld/watcom/h" ......\c\errrtns.c errrtns.c icl -nologo -c -D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS -W4 -wd4214 -MT -D_STATIC_CPPLIB -Ox -Zi -fast -Foshowopts.obj -DNDEBUG -DNT_X64 -DNT -DFLAT -DWFL__ -I. -I.. -I"../../h" -I"../../../wfc/h" -I"../../../h" -I"F:\open-watcom-v2\bld/comp_cfg/h" -I"F:\open-watcom-v2\bld/watcom/h" ......\wfc\c\showopts.c showopts.c ......\wfc\c\showopts.c(40): catastrophic error: cannot open source file "inout.h"

include "inout.h"

                ^

compilation aborted for ......\wfc\c\showopts.c (code 4) Error(E42): Last command making (showopts.obj) returned a bad status Error(E02): Make execution terminated <pmake -d build -h> => non-zero return: 2 Build failed

revelator commented 9 years ago

Looking at the include path everything seems ok -I"../../../wfc/h" and the file exists so not sure whats going wrong here ?.

jmalak commented 9 years ago

It is strange, it looks like that Intel compiler doesn't like somo construct in include path. I will try to install evaluation version of Intel compiler and check it to get idea what is wrong.

revelator commented 9 years ago

ill try leaving out the -fast option as a test, its not the first time i seen intels compiler misbehaving when certain optimization switches are used. instead ill try with seperate switches to see which work and which do not.

revelator commented 9 years ago

Hmm still not it :( guess the last option would be if my system paths are somehow screwed up.

revelator commented 9 years ago

Ok had to change the include paths manually and now it builds. One good thing the compile is ultra fast with -Qxhost -fast option :) i can build the entire thing in an hour, the slowest thing being building the documentation which takes rather long because we have to emulate a 16 bit OS.

jmalak commented 9 years ago

What exactly problem was?

revelator commented 9 years ago

f77 wfc and wfl include paths seem to not be read correctly when using the intel compiler.

revelator commented 9 years ago

first compile after your changes to the build system worked flawlessly after i did a full clean, then i updated it and it broke but so far i seen no changes to the build files for those two so im uncertain how this could have happened. It seems intels compiler does not pick up the include paths in the right order for those two directories it works everywhere else though so it might be a little snag in the build files for F77.