CLion cannot collect compiler information from owcc

[volkertb]

The owcc executable in the Open Watcom suite is the "Open Watcom C/C++ POSIX-like Compiler Driver", as the Open Watcom C/C++ Tools User’s Guide (ctools.pdf) describes it.

As I understand it, it's basically a wrapper that makes Open Watcom more compatible with the traditional software development toolchains in POSIX environments, such as GNU Make, etc.

owcc works well with CMake in combination with its default GNU Makefile generator in Linux. With owcc, WMake doesn't seem necessary for cross-platrform C/C++ development in Linux environments.

This is useful to me, since I've been trying to get Open Watcom to work with the CLion IDE from JetBrains. CLion supports only CMake projects.

Unfortunately, although I've got the CMake part working with owcc, CLion performs an additional step after building and testing the code with CMake, namely the "collection of compiler information". This works properly with the specific C/C++ compilers that CMake supports, namely GCC, Clang and I believe also the Microsoft compilers from Visual Studio in Windows environments.

As far as I could figure out, CLion performs this step to determine the default include paths of things like stdio.h, as well as the byte size of standard data types.

It seems like owcc is not sufficiently compatible with GCC and CLang.

the error that CLion shows in the console is this:

Cannot get compiler information:
    Compiler exited with error code 1: /opt/watcom/binl64/owcc -xc -bdos -fpch-preprocess -v -dD -E

By the way, the -bdos parameter was included by CLion here, because I specified it in CMakeLists.txt with the line set(CMAKE_C_FLAGS "-bdos"). If that line is omitted, it will not be included. But that doesn't matter w.r.t. the compiler information collection step, since owcc apparently fails with the other parameters as well.

owcc doesn't like the -dD parameter in particular. According to the GCC manpage, this parameter should do the following:

Dump all macro definitions, at the end of preprocessing, in addition to normal output.

After I wrote a wrapper script that strips the -dD parameter from the response file that CLion supplies to the compiler when it tries to collect compiler information, it would still fail:

Problems were encountered while collecting compiler information:
    Unexpected compiler output. This compiler might be unsupported.
    If you are using GCC/Clang, please report the bug in https://youtrack.jetbrains.com/issues/CPP.

I don't think it's just the -dD parameter, though. I suspect that owcc currently doesn't properly process the other parameters from CLion, such as -xc and -fpch-preprocess, either. Or at least it doesn't result in the same behavior and console output that CLion is looking for.

Apparently, CLion assumes a high level of compatibility with GCC command-line parameters, although CLang appears to work fine with those same parameters. It seems that CLion has to implement custom support for any other compilers and toolchains. For instance, in the just very recently released version 2020.1 of CLion, support for the IAR Toolchain (a commercial product for embedded software development) was announced:

If you use the IAR compiler/toolchain in your embedded projects, you can now do so in CLion. Collecting compiler information no longer fails, which means projects using the IAR toolchain load successfully and work in CLion.

(Emphasis mine)

By the way, so far I've tried this with versions 2019.3.5 and 2020.1 of CLion, with the same (failing) results.

So debugging this further (again, by writing a wrapper script and configuring it as the compiler executable), I discovered that CLion does the following under the hood when it tries to "collect the compiler information":

• It creates two temporary files, one /tmp/response-file[RANDOM_NUMBER].txt and the other /tmp/compiler-file[A_DIFFERENT_RANDOM_NUMBER] (with actual numbers in the place of those bracketed placeholders)
• It runs owcc @/tmp/response-file[RANDOM_NUMBER].txt /tmp/compiler-file[A_DIFFERENT_RANDOM_NUMBER] (with the numeric parts differing on each invocation, likely the current timestamp)
• CLion deletes those temporary files (I used the wrapper script to copy them to another location for analysis)

The response file is a standard mechanism for providing a higher number of arguments than the current platform may support on the command-line. The file contains the following parameters:

-xc -bdos -fpch-preprocess -v -dD -E -D___CIDR_DEFINITIONS_END

Again, the -bdos argument (for cross-compiling 16-bit DOS code) will only be there if specified as a custom C compiler parameter in the CMake lists file. The compiler information collection will fail eitherway.

The source file it feeds to the compiler as the second argument has the following contents:

compiler-file.txt

The response that owcc throws out on the console is the following:

Result:
Error! E1073: Invalid option '-xc'
Error! E1073: Invalid option '-bdos'
Error! E1073: Invalid option '-fpch'
Error! E1073: Invalid option '-preprocess'
Error! E1147: Too many errors: compilation aborted
Error: Compiler returned a bad status compiling '@/tmp/response-file510283241670491082.txt'
Error! Unable to open '/tmp/compiler-file16295087320389008726.c'
Error: Compiler returned a bad status compiling '/tmp/compiler-file16295087320389008726'

As you can see, owcc suffixed a .c extension to the compiler-file without asking.

Strangely enough, the owcc compiler would return an exit code 0 here, causing CLion to simply complain about unexpected compiler output, instead of showing the actual errors. (I made the wrapper script log this to a separate file, to get these erors.)

Also, equally weirdly, owcc is complaining about the -bdos parameter here, which it most definitely should support.

After enhancing the wrapper script to automatically create a symlink to the compiler-file with the same name, but with the .c extension, the compiler would instead complain about the file not containing proper C code.

Long story short, owcc doesn't support the collection of compiler information in the way that CLion attempts it, which apparently does work well with gcc and clang.

As a result of this step failing, CLion is unable to determine things like include path resolution, which results in errors in the IDE like this:

Proper code highlighting and contextual support are exactly the reason why I wanted to use CLion in the first place. :disappointed:

Is this a known issue? And how easy would this be to fix? So that owcc would correctly process these parameters and input files that CLion supplies when it tries to collect compiler information from it, and then respond with the output that CLion would expect and would need to correctly configure the project?

Thank you for the hard work you've been doing to maintain Open Watcom for all us hobbyists and legacy software developers! :slightly_smiling_face:

[jmalak]

Take into acount that owcc is driver which emulate posix-like command line not all features (see POSIX specification for cc) and has nothing to do with gcc and clang drivers and theirs functionality even if use some of gcc like parameters. OW is natively cross-compiler and is rather similar to clang native mode then gcc or gcc-emulation. But this cross-compile features are not based on any prefix or suffix for some path as with gcc or clang. Also not all options behave the same way as in gcc or clang, OW uses totaly different aproach. If CLion has some expectation you must specify what is realy sending to owcc and what is expected to get back. Is CLion communication with OW directly or only with CMake? Without these info I can not do anything. Anyway owcc is not any replacement or emulation of gcc or clang. We have no plan to follow clang or gcc any way. OW is independent and has no sense to do anything like gcc or clang that you can not relate to any internal gcc and clang features.

[jmalak]

only way to use OW with CMake is real cross-compile stuff for OW in CMake and implement features required by CLion. But it is hard because CLion is closed code. CLion or CMake must use OW possibility not gcc or clang if want to get any info. It is something like you want to use Visual studio by gcc or clang stuff, it is simply impossible.

[jmalak]

anyway command line like -xc -bdos -fpch-preprocess -v -dD -E -D___CIDR_DEFINITIONS_END is wrong because owcc uses POSIX rules that correct should be -x c -b dos -f pch-preprocess -v -d D -E -D ___CIDR_DEFINITIONS_END

[volkertb]

Also not all options behave the same way as in gcc or clang, OW uses totaly different aproach.

That's what I was afraid of.

If CLion has some expectation you must specify what is realy sending to owcc and what is expected to get back.

I can try to save a dump of what gcc and clang send back when CLion queries those compilers for compiler information like it tries to do with owcc.

Is CLion communication with OW directly or only with CMake?

So CLion appears to be performing two steps here. The first step is to tell CMake to compile a test program, to make sure the toolchain is sane. That was the part that failed when I tried to use the WMake generator of CMake, which led me t open issue #594. With owcc and the default generator (instead of wcc and the WMake generator), this part succeeds.

The second step is the "collecting of compiler information", which CLion attempts by sending the compiler some parameters and input directly (apparently without going through CMake) and by parsing the output it gets bvack from the compiler. This is the part that fails here, since owcc does not accept all the parameters/arguments that CLion sends to it.

We have no plan to follow clang or gcc any way. OW is independent and has no sense to do anything like gcc or clang that you can not relate to any internal gcc and clang features.

Looks like there's no easy solution to get owcc working seamlessly with CLion, then. :disappointed:

anyway command line like -xc -bdos -fpch-preprocess -v -dD -E -D___CIDR_DEFINITIONS_END is wrong because owcc uses POSIX rules that correct should be -x c -b dos -f pch-preprocess -v -d D -E -D ___CIDR_DEFINITIONS_END

I tried that, but it still complains about Invalid option 'd'. And even when I remove the -d D part, it doesn't seem to process the other parameters as intended either.

[jmalak]

I think all misunderstanding is that owcc is gcc emulator. No it is wrong expectation. owcc is POSIX cc compatible. If CMake and CLion support POSIX cc then it could work. You can not relate to gcc or clang compiler feature because owcc is not emulator of them.

[jmalak]

Anyway owcc doesn't support -d option (non-POSIX) that the message is correct. options supported by owcc and syntax can be displayed by command owcc -?

Usage:  owcc [options] file ...

-c                              compile only, no link
-x {c,c++}                      treat source files as C/C++ code
-o <name>                       set output file name
-b <target>                     compile and link for target
-v                              show sub program invocations

                [Processor options]
-march=i{,1,2,3}86,axp,mips,ppc generate code for this architecture
-mtune=i{3,4,5,6}86             optimize for this family
-mregparm=<n>                   calling convention

                [x86 Floating-point processor options]
-fpmath={,2,3}87                hardware floating-point code
-fptune={5,6}86                 optimize f-p for CPU type
-msoft-float                    calls to floating-point library

                [Compiler options]
-mcmodel={f,t,s,m,c,l,h}        memory model
-mabi=c                         set calling conv. to __cdecl
-mabi=d                         set calling conv. to __stdcall
-mabi=f                         set calling conv. to __fastcall
-mabi=p                         set calling conv. to __pascal
-mabi=r                         set calling conv. to __fortran
-mabi=s                         set calling conv. to _System
-mabi=w                         set calling conv. to __watcall
-mwindows                       generate code for MS Windows
-shared                         generate code for a DLL
-fno-short-enum                 force enums to be type int
-fshort-enum                    use minimum base type for enum
-fsigned-char                   change char default to signed
-fpack-struct={1,2,4,8,16}      default struct member alignment
-fno-rtti                       (C++) disable RTTI
-std={c89,c99,wc}               disable/enable extensions

-Wlevel<n>                      set warning level number
-Wall                           set warning level to 4
-Wextra                         enable maximal warnings
-Werror                         treat all warnings as errors
-Wn<n>                          enable warning message <n>
-Wno-n<n>                       disable warning message <n>
-Woverlay                       warn about problems in overlaying
-Wpadded                        warn when padding a struct
-Wstop-after-errors=<n>         stop compiler after <n> errors
-fmessage-full-path             full paths in messages
-femit-names                    emit routine names in the code
-fbrowser                       generate browsing information

-fhook-epilogue                 call epilogue hook routine
-fhook-prologue[=<n>]           call prologue hook routine
-fo=<ext>                       set object file extension
-fr=<file>                      output errors to a file name
-fwrite-def                     output func declarations to .def
-fwrite-def-without-typedefs    same, but skip typedef names
-fno-stack-check                remove stack overflow checks
-fgrow-stack                    generate calls to grow the stack
-fstack-probe                   touch stack through SS first
-fno-writable-strings           don't place strings in CODE segment
-fnostdlib                      remove default library information
-ffunction-sections             functions in separate segments
-fvoid-ptr-arithmetic           allow arithmetic operations on (void *)
-fnonconst-initializers         allow non-constant initializers
-fsyntax-only                   check syntax only
-Wc,<option>                    pass any option to WCC

                [Debugging options]
-g0                             no debugging information
-g1{+}                          line number debugging info.
-g2                             full symbolic debugging info.
-g2i                            (C++) + inlines as COMDATs
-g2s                            (C++) + inlines as statics
-g2t                            (C++) without type names
-g3                             add info for unreferenced type names
-g3i                            (C++) + inlines as COMDATs
-g3s                            (C++) + inlines as statics
-gcodeview                      codeview debug format
-gdwarf                         dwarf debug format
-gwatcom                        watcom debug format

                [Optimization options]
-O0                             disable optimizations
-O{1,2,3}                       optimization presets
-Os                             optimize for space
-Ot                             optimize for time

-fno-strict-aliasing            relax alias checking
-fguess-branch-probability      branch prediction
-fno-optimize-sibling-calls     disable call/ret optimization
-finline-functions              expand functions inline
-finline-limit=num              which functions to expand inline
-fno-omit-frame-pointer         generate traceable stack frames
-fno-omit-leaf-frame-pointer    generate more stack frames
-frerun-optimizer               enable repeated optimizations
-finline-intrinsics[-max]       inline intrinsic functions
-fschedule-prologue             control flow entry/exit seq.
-floop-optimize                 perform loop optimizations
-funroll-loops                  perform loop unrolling
-finline-math                   generate inline math functions
-funsafe-math-optimizations     numerically unstable floating-point
-ffloat-store                   improve floating-point consistency
-fschedule-insns                re-order instructions to avoid stalls
-fkeep-duplicates               ensure unique addresses for functions

-feh[=s,=t]                     exception handling: balanced, space, time

                [Preprocessor options]
-D<name>[=text]                 define a macro
-D+                             extend syntax of -D option
-U<name>                        undefine macro name
-I <directory>                  include directory
-include <file>                 force pre-include of file
-ftabstob=<n>                   # of spaces in tab stop
-tp=<name>                      (C) set #pragma on <name>
-pil                            ignore #line directives
-E                              preprocess source file
-C                              preserve comments
-P                              don't insert #line directives
-fcpp-wrap=<n>                  wrap output at column n
-fmangle-cpp                    encrypt C++ names
-M[M]D                          output autodepend make rule
-MF <file>                      autodepend output name
-MT <target>                    autodepend target name

                [Linker options]
-mthreads                       build Multi-thread application
-mrtdll                         build with dll run-time library
-fd[=<file>]                    save WLINK directives to file
-fm[=<map_file>]                generate map file
-mstack-size=<stack_size>       set stack size
-s                              strip debug info from executable
-Wl,<option>                    pass any directive to WLINK

[claunia]

@jmalak should it be possible to make owcc accept the parameters and generate the output clion sends/expect as gcc and clang do?

[jmalak]

I think it is CMake functionality that CMake must support this for OW toolchain. We don't plan to do any emulation for gcc or clang features. If CLion use CMake then it is about CMake support for OW toolchain and it requires to use OW tools possibility not over gcc or clang emulation. We don't want to develop another gcc or clang clone. It is necessary to know what CMake features is used by CLion to be able implement it in CMake for OW toolchain.

[jmalak]

Anyway it is rather discussion for CMake development forum then for OpenWatcom, because it is CMake tool problems. We can help you with OpenWatcom toolchain information or explanation of OpenWatcom functionality, but CMake tool must be fixed by CMake developpers to support appropriate features for OpenWatcom.

[jmalak]

We shloud move this discussion to CMake development forum. I think it is relevant for this problem.

[volkertb]

No, this part is not CMake-related. It's the "collecting compiler information" phase in which CLion tries to invoke the compiler correctly.

[jmalak]

In this case it is problem of CLion not OW, CLion expect something what is not true. What is problem exactly, what is wrong with C/C++ compiler? You say that OW is wrong, but not specify what is wrong exactly.

[volkertb]

I guess I was hoping that the command-line arguments and compiler outputs that CLion's "collecting compiler information" functionality depended on would all be standard generic POSIX compiler behavior. Apparently that's not the case.

Indeed, this probably means that CLion would have to add specific support for the Open Watcom compiler. If only there were a more standardized way for IDEs to interact with compilers to query their functionality and platform-dependent properties. :disappointed:

I guess there's not much else to do that to open a feature request at JetBrains for adding Open Watcom (or at least owcc) support to CLion.

Does anybody else here have any suggestions? Perhaps for some kind of workaround? Or shall I just close this issue, then?

[volkertb]

Hmmm... This guy has been working on a workaround for a similar problem with another unsupported compiler: https://intellij-support.jetbrains.com/hc/en-us/community/posts/360003275680-How-to-make-CLion-recognise-my-fake-compiler-in-compilation-database-

Apparently he wasn't successful yet, but maybe his work could form a basis for a more generic solution that could eventually perhaps also benefit Open Watcom.

Anyway, that's outside the scope of this issue, which indeed is not the responsibility of the Open Watcom developers to solve. So I'm closing this issue.

Nevertheless, thanks for all your feedback and clarifications!

[jmalak]

CMake has some support for compilers feature. If it is used by CLion then it could be fixed by adding this feature support for OW to CMake. I think it is not only problem with compilers it could be a problem with debugger too. CLion support only GDB and LLDB, that it can not works with OW debugger. You should use some debugger format used by supported Debuggers. OW support Dwarf V2 (but there is a few OW extensions related to segmented architecture) and CodeView V4 which could be compatible with some supported debuggers, but I am not sure how these debuggers refere to GNU/gcc specific things etc.

[volkertb]

Debugger support would be a nice second step, but to begin with, I would like to have a functional IDE with working code highlighting, auto-completion and basic refactoring (renaming, etc) working. But yeah, a standardized cross-platform and vendor-independent way to determine compiler and debugging features would be the key here.

I wonder why CLion doesn't use CMake for collecting the necessary "compiler information". Apparently CMake's functionality in that regard is inadequate.

I don't have experience with other modern popular cross-platform C/C++ IDEs. Do any of you know how those handle this? And how well those work with Open Watcom? I mean IDEs like Eclipse, NetBeans, Visual Studio Code, etc?

[volkertb]

Cool! I was thinking about opening an issue in JetBrains YouTrack, but you beat me to it, @jmalak . Good job. :sweat_smile:

https://youtrack.jetbrains.com/issue/CPP-20108

Please upvote this issue, people! :+1:

[volkertb]

@jmalak Perhaps you could also upvote this issue? Thanks. :slightly_smiling_face: https://youtrack.jetbrains.com/issue/CPP-20109

[jmalak]

I think that the main problem is with multi-platform and cross-platfom difference. Most of such tools are only multi-platform for host not for target. pure cros-platform tools requires remote debugging for toolchain. OW is very different than GNU like tools. OW is by nature cross-platform and don't need similar configuration/installation as GNU tools with some prefixes and suffixes. Only OW compilers is one architecture that you need call correct compiler for architecture , but all is available on all host system by default. librarian, linker and other tools supports all platforms by nature. OW can be switch by a few environment variables to any supported target. All OW tools are native doesn't require any emulation of POSIX layer or UNIX like environment as by example CygWin etc.

[claunia]

Wouldn't just this fix the problem?:

diff --git a/bld/wcl/c/owcc.c b/bld/wcl/c/owcc.c
index 3552391c3..cac13dc0d 100644
--- a/bld/wcl/c/owcc.c
+++ b/bld/wcl/c/owcc.c
@@ -684,11 +684,11 @@ static  int  ParseArgs( int argc, char **argv )
     AltOptChar = '-'; /* Suppress '/' as option herald */
     while( (i = GetOpt( &argc, argv,
 #if 0
-                        "b:Cc::D:Ef:g::"
+                        "b:Cc::dD:Ef:g::"
                         "HI:i::L:l:M::m:"
                         "O::o:P::QSs::U:vW::wx:yz::",
 #else
-                        "b:CcD:Ef:g::"
+                        "b:CcdD:Ef:g::"
                         "HI:i::L:l:M::m:"
                         "O::o:P::QSs::U:vW::wx::yz::",
 #endif

Just ignoring the -dD suffices to make CLion relatively happy.

[volkertb]

Okay, the CLion developers are currently working on custom compiler toolchain support, including Open Watcom. :slightly_smiling_face:

See YouTrack issue https://youtrack.jetbrains.com/issue/CPP-9615

Fingers crossed. :crossed_fingers:

[claunia]

Not wanting to resurrect a zombie but: https://blog.jetbrains.com/clion/2021/10/clion-2021-3-eap-custom-compiler/

and a first try, can be made better, here: https://github.com/JetBrains/clion-custom-defined-compiler-examples/issues/2 https://github.com/JetBrains/clion-custom-defined-compiler-examples/issues/3 https://github.com/JetBrains/clion-custom-defined-compiler-examples/issues/4 https://github.com/JetBrains/clion-custom-defined-compiler-examples/issues/5 https://github.com/JetBrains/clion-custom-defined-compiler-examples/issues/6

[volkertb]

Good to see that this feature will finally be included in Clion 2021.3. :relaxed:

And thanks for creating Open Watcom yaml files for it, @claunia!

I'll try some of them out, and I'll share my findings with you.

[claunia]

A little how to use: CLion uses cmake. Current CLion EAP includes the CMake patches (@jmalak ?) submitted for OpenWatcom. So to make it work for DOS you need to add in Build, Execution, Deployment -> Toolchains the compiler so for C and C++, the wcc or wcc386. Then in the project settings, in CMake you need to add -G "Unix Makefiles" -DCMAKE_SYSTEM_NAME=DOS, that variable CMAKE_SYSTEM_NAME is what will tell CMAKE with -bt to use, I only tested DOS and OS2 and seen that Windows is a valid one (that will set -bt=dos, -bt=os2 and -bt=nt respectively) but how to set the other targets (NetWare, RDOS, DOS/4GW, Win32s) no idea.

[volkertb]

@claunia Are you running CLion and the Open Watcom cross-compiler from Windows, or macOS?

~Because I cannot for the love of me get this to work under Linux. And according to the Arch Linux wiki, The WMake generator of CMake still can't be used in Linux to set up cross compilation. It apparently only works on Windows.~

~That's why I resorted to using owcc with the GNU Makefile generator. I got that working earlier a while back, but I'm getting problems getting that working again.~

(See comments below.)

[jmalak]

What problem do you have with CMake on Linux? The use of CMake with OW suppose that you have OW configured for host system only (in your case for Linux host/target).

[volkertb]

@jmalak Oh, sorry, forget my previous comment. You already fixed CMake and WMake to be able to work together for cross-compiling DOS executables from Linux, a while back. as you mentioned here.

Looks like the Arch Linux wiki is just not updated with this new information yet.

So I have it working with CMake on the command-line now. It's just that I keep getting errors when I want to get this working in CLion 2021.3 EAP (Early Access Preview) as well. @claunia already got it working.

[jmalak]

OK. I will add page to OW Wiki with simple description of using OW with CMake for cross-compilation, to be useful for other OW users.

[volkertb]

@claunia Can you maybe share with us exactly what you have configured in Build, Execution, Deployment -> Toolchains?

It seems that whatever I specify, I always get a warning icon with the message "Test CMake finished with errors", with a "more..." link I could click on, which would lead to an console log with errors that don't seem to show a clear cause.

What values did you enter for the following:

• Make:
• C Compiler:
• C++ Compiler:

Additionally, did you configure any environment variables anywhere in CLion? (WATCOM, INCLUDE, additions to the PATH, etc)?

Also, with -G "Unix Makefiles" -DCMAKE_SYSTEM_NAME=DOS you are using the Unix Makefile generator as opposed to the (Open) Watcom Wmake generator. And you're using it with wcc and wpp, seemingly? I never got that to work.

At least from the command-line, either I could make it work with the WMake generator combined with wcc and wpp, or with the Unix Makefile generator combined with owcc.

But in CLion, so far, I've gotten neither of the two to work.

Your help is appreciated!

[volkertb]

@claunia Particularly, how did you get around the following errors, every time CLion tried to "compile a simple test program", using both regular Unix/GNU Make and wcc/wpp?

That's what I keep running into, whenever I try that combination of make tool and compiler with CLion 2021.3 EAP:

Error! E1073: Invalid option '-c'
Error! E1073: Invalid option '-cc'

It's so frustrating how hard it is to get this working. But I really want to use CLion for my retro C projects! I'm just so comfortable using JetBrains IDEs.

If it wasn't for the fact that your posts implied that you got it working, I would have concluded that the new custom compiler support feature in CLion isn't fully working yet.

[claunia]

@volkertb I did a quick and dirty try in CLion for Windows. Do not despair, it will work, JetBrains is very collaborative and my main system is ArchLinux, as soon as I have time I'll take a deeper look.

[claunia]

Ok I checked on Linux and I can see what you say, this is the command line being used by CMake to test the compiler:

/opt/watcom/binl64/wcc -zq -d+  -I/h -w3 -bt=dos  -d2 -foCMakeFiles/cmTC_991a8.dir/testCCompiler.c.obj -c -cc /path/to/CMakeFiles/CMakeTmp/testCCompiler.c

I can see several differences here, like for example -I/h would tell the compiler to use /h as an include directory, then it is adding -c -cc.

This is CMake, not CLion, so @jmalak any idea here?

[claunia]

Yeah this has nothing to do with CLion:

claunia@zeus ~/Development/Aaru/fstester/setter $ cmake -DCMAKE_C_COMPILER=/opt/watcom/binl64/wcc -DCMAKE_CXX_COMPILER=/opt/watcom/binl64/wpp .
CMake Deprecation Warning at CMakeLists.txt:25 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- The C compiler identification is OpenWatcom 2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: /opt/watcom/binl64/wcc
-- Check for working C compiler: /opt/watcom/binl64/wcc - broken
CMake Error at /usr/share/cmake-3.21/Modules/CMakeTestCCompiler.cmake:69 (message):
  The C compiler

    "/opt/watcom/binl64/wcc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /home/claunia/Development/Aaru/fstester/setter/CMakeFiles/CMakeTmp

    Run Build Command(s):/usr/bin/make -f Makefile cmTC_d4f5b/fast && /usr/bin/make  -f CMakeFiles/cmTC_d4f5b.dir/build.make CMakeFiles/cmTC_d4f5b.dir/build
    make[1]: se entra en el directorio '/home/claunia/Development/Aaru/fstester/setter/CMakeFiles/CMakeTmp'
    Building C object CMakeFiles/cmTC_d4f5b.dir/testCCompiler.c.o
    /opt/watcom/binl64/wcc -zq -d+   -w3 -bt=linux  -d2 -foCMakeFiles/cmTC_d4f5b.dir/testCCompiler.c.o -c -cc /home/claunia/Development/Aaru/fstester/setter/CMakeFiles/CMakeTmp/testCCompiler.c
    Error! E1073: Invalid option '-c'
    Error! E1073: Invalid option '-cc'
    make[1]: *** [CMakeFiles/cmTC_d4f5b.dir/build.make:78: CMakeFiles/cmTC_d4f5b.dir/testCCompiler.c.o] Error 1
    make[1]: se sale del directorio '/home/claunia/Development/Aaru/fstester/setter/CMakeFiles/CMakeTmp'
    make: *** [Makefile:127: cmTC_d4f5b/fast] Error 2

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:27 (project)

-- Configuring incomplete, errors occurred!
See also "/home/claunia/Development/Aaru/fstester/setter/CMakeFiles/CMakeOutput.log".
See also "/home/claunia/Development/Aaru/fstester/setter/CMakeFiles/CMakeError.log".

[claunia]

Yes confirmed, here: https://gitlab.kitware.com/cmake/cmake/-/blob/master/Modules/Compiler/OpenWatcom.cmake#L59

That command line is not working with wcc.

Removing it:

launia@zeus ~/Development/Aaru/fstester/setter $ cmake -DCMAKE_C_COMPILER=/opt/watcom/binl64/wcc386 -DCMAKE_CXX_COMPILER=/opt/watcom/binl64/wpp386 .
CMake Deprecation Warning at CMakeLists.txt:25 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- The C compiler identification is OpenWatcom 2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/watcom/binl64/wcc386 - skipped
Detected system name: Linux
Detected system processor: x86_64
-- Looking for statfs
-- Looking for statfs - not found
-- Looking for statfs
-- Looking for statfs - not found
-- Looking for statvfs
-- Looking for statvfs - not found
-- Looking for statfs
-- Looking for statfs - not found
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_CXX_COMPILER

-- Build files have been written to: /home/claunia/Development/Aaru/fstester/setter

However when using cmake -DCMAKE_SYSTEM_NAME=DOS -DCMAKE_C_COMPILER=/opt/watcom/binl64/wcc -DCMAKE_CXX_COMPILER=/opt/watcom/binl64/wpp -G "Unix Makefiles" . as I did on Windows, it is not respecting the cmake system name: /opt/watcom/binl64/wcc -zq -d+ -w3 -bt=linux -d2 -foCMakeFiles/cmTC_bbc8c.dir/testCCompiler.c.o /home/claunia/Development/Aaru/fstester/setter/CMakeFiles/CMakeTmp/testCCompiler.c

As you see it's building for Linux, so wlink then fails miserably:

/opt/watcom/binl64/wlink option quiet name cmTC_bbc8c opt map system linux opt noextension  debug all file {CMakeFiles/cmTC_bbc8c.dir/testCCompiler.c.o} 
    Warning! W1080: file CMakeFiles/cmTC_bbc8c.dir/testCCompiler.c.o is a 16-bit object file
    Warning! W1008: cannot open emu87.lib : No such file or directory
    Warning! W1008: cannot open math87s.lib : No such file or directory
    Warning! W1008: cannot open clibs.lib : No such file or directory
    Error! E2028: __STK is an undefined reference
    Error! E2028: _cstart_ is an undefined reference
    Error! E2028: _small_code_ is an undefined reference
    Error! E2028: __argc is an undefined reference
    Warning! W1023: no starting address found, using 00000000
    Error! E2044: exported symbol __STK not found
    Error! E2044: exported symbol _cstart_ not found
    Error! E2044: exported symbol _small_code_ not found
    Error! E2044: exported symbol __argc not found

[jmalak]

I wrote several times CMake use standard OW setup and don't need specify path to compilers it is defined by PATH and don't change any way. OW must be always setup for host system even if you will cross-compile to other platform.

• WATCOM environment variable pointing to the root of OW installation
• PATH must include subirectory with host binaries
• INCLUDE to point to host header files directory Nothing more, if you specify compiler location it is useless and can do more problems, all binaries are on the PATH and such setup can be only source of problem. All is handled by CMake (handle header files for selected target, compiler options etc.)

!!!!!! all the problems are due to use specify wcc compiler but CMake use wcl tool which has different options !!!!!!

!!!!! dont secify OW compiler anyway, it will be done properly by CMake. !!!!!!

Anyway wcl is driver which handle all OW tools C/C++/assembler/Fortran compiler and linking for all such oject file together to executable. wcc and wpp are compilers with their own syntax which is not same as for wcl driver. I thing it is problem they people want configure OW toolchain wrong way even if it doesnt require any. Cmake handle all this by two parameters CMAKE_SYSTEM_NAME and CMAKE_SYSTEM_PROCESSOR nozhing more.

[claunia]

CMake can use standard OW setup but CLion expects you to tell the CMAKE_C_COMPILER and CMAKE_CXX_COMPILER so let's go step by step right now, trying using wcl instead of wcc.

We can also tell CLion to define environment variables, as well as cmake system name and processor, so I'm flying blind here, I didn't know of wcl existence and I'm just trying things until everything works.

[claunia]

GOTCHA!

Ok, in Windows, this is what I tried: PATH, as included in owsetenv.bat, set in the system preferences so it is global, there seems to be no way to set it up from CLion.

On my yaml, change wcc for wcl.

The on CLion -> Build etc -> Toolchains: Make: Pick wmake.exe C compiler: Pick wcl.exe C++ compiler: Pick wcl.exe

Then on CLion -> Build etc -> CMake Create a new one, choose the toolchain and then put: CMake options: -DCMAKE_SYSTEM_NAME=DOS -DCMAKE_SYSTEM_PROCESSOR=I86 -G "Watcom WMake" Doesn't compile properly if you leave as is (will use Ninja) And in environment I added: WATCOM=C:\WATCOM and INCLUDE=C:\WATCOM\H;C:\WATCOM\H\NT;C:\WATCOM\H\NT\DIRECTX;C:\WATCOM\H\NT\DDK;%INCLUDE%

And got it:

[ 20%] Linking C executable fssetter-DOS-I86.exe
[100%] Built target fssetter-DOS-I86

Going to try other targets now and then I'll try on Linux, but great first steps here people.

[claunia]

Ok Ok ,all this on Wndows but now got it working with the following differences, only in CMake options. -DCMAKE_SYSTEM_NAME=DOS -DCMAKE_SYSTEM_PROCESSOR=I86 -G "Watcom WMake" -DCMAKE_SYSTEM_NAME=OS2 -DCMAKE_SYSTEM_PROCESSOR=I86 -G "Watcom WMake" -DCMAKE_SYSTEM_NAME=OS2 -DCMAKE_SYSTEM_PROCESSOR=I386 -G "Watcom WMake" -DCMAKE_SYSTEM_NAME=Windows -DCMAKE_SYSTEM_PROCESSOR=I386 -G "Watcom WMake"

For I386 using wcl386 obviously. I'll try to do a hello.c in all targets to confirm before trying in Linux.

[claunia]

Ok first revision, with instructions: https://github.com/JetBrains/clion-custom-defined-compiler-examples/pull/7.

[jmalak]

Please don't setup compiler executable. I don't understand why you want to do this, it is the source of problems?

Anyway steps in CLion doesn't depend on host if you are on Windows or Linux it should be same. CMake and wmake handle forward and backward slash according to host,. Preferred is forward slash to have one version only for both hosts. If it doesn't work properly report it to be fixed.

[claunia]

You MUST setup the compiler because if you leave it empty, CLion still sets it up to a default, and fails.

What works when cmake is invoked alone doesn't necessarily work when cmake is invoked by clion. If we remove the environment variables from CLion configuration things break: Warning! W1115: system block dos: environment name WATCOM not found.

With those instructions everything I tested works perfectly, and the only things missing are:

• How to tell CMake to compile 16-bit Windows (I have not even tried it).
• How to tell CMake to compile NetWare (doesn't seem to be a Novell-OpenWatcom.cmake file).
• How to compile QNX, Alpha, MIPS or PowerPC at all (I know C libraries are missing but people may want to compile without it).

[jmalak]

I understand. CMake OW support is not for all targets. I introduced DOS and OS/2 targets to existing Windows and Linux because these didn't exist before in CMake. Unfortunately CMake OW support can be extended, but has no sense for something not used. QNX OS and Alpha, MIPS and PowerPC are experimntal and are not supported. For such target you must configure all manualy for CMake through cross-compile file and it must be specified to CMake. I am not sure if I added support for 16-bit Windows try specify as -DCMAKE_SYSTEM_NAME=Windows -DCMAKE_SYSTEM_PROCESSOR=I86

[volkertb]

Pfew! I finallly got it working in CLion 2021.3 EAP in Linux, using @claunia's instructions posted further above, with the following additional caveats:

• In addition to setting the WATCOM and INCLUDE environment variables, I also had to add /opt/watcom/binl64 to the PATH so wcl could find wcc. The hint for this could be found in CMakeFiles/CMakeError.log, wich contained the line Error: Unable to find 'wcc'. I did this by setting PATH to in the environment settings under the CMake settings in CLion. By the way, $WATCOM/binl64:$PATH didn't work there. I had to use the absolute path. I later found it to be easier to just add the line . /opt/watcom/owsetenv.sh to the ~~/.profile~ ~/.bashrc file of my Linux user account instead. That way, it would also always just work from the command line, in addition to being picked up by CLion automatically.
• I had to delete CMakeCache.txt after fixing this, because the error was basically being cached by CMake. I've hit my head against this stupid issue before. Why does CMake cache build errors? :confused:
• The ⚠️ Test CMake run finished with errors message in the Toolchains settings will always be shown after configuring the toolchain, and can apparently be ignored. The output contains a lot of errors such as Makefile(115): Error(E21): Extension(s) (.PHONY) not defined, and this is likely due to CLion's test run defaulting to the Unix Makefile generator, resulting in a Makefile that wmake can' process. It's not a blocking issue, but it might be useful to report this on YouTrack upstream.

@claunia @jmalak Thank you both so very much for helping me get this working in CLion! Now I can hopefully do some retro DOS tinkering from the comfort of a familiar JetBrains IDE. :relaxed:

[claunia]

@volkertb I'm pretty sure I wrote in the documentation that you need to set the PATH system-wide. I tried doing it in CLion to no avail.

Yes CMakeCache.txt needs to be removed every now and then.

And your test failed with errors, no idea, are you following the instructions here ? These are complete and tested instructions for both Windows and Linux. CLion EAP defaults to Ninja generation from CMake, you need to specify Watcom WMake generation.

[volkertb]

Yeah, I initially missed the system-wide PATH step. It wasn't until after I figured that out that I noticed that at the top of your instructions. I should have read that better the first time.

By the way, it wasn't my test that failed. It's the sanity test that CLion performs automatically whenever you configure a toolchain. It's just the sanity test that fails, though. Other than that, everything works fine.

Aren't you getting the same error?

See this screenshot, particularly the error right below the C++ Compiler: field:

And when you click on more..., this is the error log that is shown:

  The C compiler

    "/opt/watcom/binl64/wcl"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /tmp/cmake_check_environment/_build16533429417646465354/CMakeFiles/CMakeTmp

    Run Build Command(s):wmake -f Makefile cmTC_a7dae/fast && Open Watcom Make Version 2.0 beta Oct 17 2021 00:52:40 (64-bit)
    Copyright (c) 2002-2021 The Open Watcom Contributors. All Rights Reserved.
    Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
    Source code is available under the Sybase Open Watcom Public License.
    See http://www.openwatcom.org/ for details.
    Makefile(6): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(9): Error(E21): Extension(s) (.NOTPARALLEL) not defined
    Makefile(18): Warning(W18): Unrecognized or out of place character ','
    Makefile(24): Warning(W18): Unrecognized or out of place character ','
    Makefile(38): Warning(W18): Unrecognized or out of place character '='
    Makefile(45): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(75): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(79): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(85): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(89): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(96): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(101): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(105): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(110): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(115): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(123): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(128): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(131): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(136): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(139): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(144): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(147): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(152): Error(E21): Extension(s) (.PHONY) not defined
    Makefile(165): Error(E21): Extension(s) (.PHONY) not defined
    Error(E02): Make execution terminated

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)

-- Configuring incomplete, errors occurred!
See also "/tmp/cmake_check_environment/_build16533429417646465354/CMakeFiles/CMakeOutput.log".
See also "/tmp/cmake_check_environment/_build16533429417646465354/CMakeFiles/CMakeError.log".

Error code: 1

[claunia]

Absolutely not getting that error. Did you setup the generator as Watcom WMake with -G ?

[volkertb]

If you mean the CMake options: under Build, Execution, Deployment > CMake: yes. I have that set to -DCMAKE_SYSTEM_NAME=DOS -DCMAKE_SYSTEM_PROCESSOR=I86 -G "Watcom WMake", as you instructed, and my project doesn't build without it.

But the "single test program" that is generated and (re)run whenever I change anything in Build, Execution, Deployment > Toolchains is apparently still being built with the default (non-wmake) generator. Did you specify an environment file in the toolchain? I've left that field empty.

[volkertb]

I tried export CMAKE_GENERATOR="Watcom WMake" before running CLion. I also tried replacing the bundled CMake with al external version. Neither worked to resolve the ⚠️ Test CMake run finished with errors message in Build, Execution, Deployment > Toolchains.

But since it works fine otherwise, I guess I can live with it. Still weird how it's apparently not an issue in your environment. Perhaps this error only occurs in Linux for some reason?

[claunia]

No, I didn't, and neither did I set up wmake for CLion, that's only needed in Windows where there's no default make:

Possibly CLion is trying to do something with make, even if it's not used, CMake knows it needs to call wmake.

[volkertb]

That's it! Leaving the Make: field empty in the Toolchain configuration made the error disappear for me as well.

Thanks again! :slightly_smiling_face: