Errors using cquery on Windows

[andyleejordan]

Hello,

I am trying to get cquery up and running for Mesos on Windows. I've succesfully built cquery and got Emacs setup to use it; and I have Mesos successfully building with Ninja on Windows (which lets me use the CMake generated compile_commands.json. However, when cquery tries to index one of my files, I get this error:

2018-02-17 21:42:06.474 (   4.013s) [indexer2     ]clang_translation_unit.cc:117   libclang had ast read error for C:/Users/andschwa/src/mesos/src/tests/utils.cpp. Please try running the following, identify which flag causes the issue, and report a bug. cquery will then filter the flag for you  automatically:
$ /TP -working-directory C:/Users/andschwa/src/mesos/build -xc++ -std=c++14 -DBUILD_DIR="C:/Users/andschwa/src/mesos/build" -DCURL_STATICLIB -DGOOGLE_GLOG_DLL_DECL= -DHAVE_LIBZ -DLIBDIR="WARNINGDONOTUSEME" -DLIBSASL_EXPORTS=1 -DNOGDI -DNOMINMAX -DPICOJSON_USE_INT64 -DPKGDATADIR="" -DPKGLIBEXECDIR="WARNINGDONOTUSEME" -DPKGMODULEDIR="WARNINGDONOTUSEME" -DSBINDIR="WARNINGDONOTUSEME" -DSOURCE_DIR="C:/Users/andschwa/src/mesos" -DTESTLIBEXECDIR="WARNINGDONOTUSEME" -DUSE_CMAKE_BUILD_CONFIG -DUSE_STATIC_LIB -DVERSION="1.6.0" -DZOOKEEPER_VERSION="3.4.8" -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_WARNINGS -D_SCL_SECURE_NO_WARNINGS -D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING -D__STDC_FORMAT_MACROS -D__WINDOWS__ -I..\include -Iinclude -Iinclude\mesos -Isrc -I..\src -I..\3rdparty\libprocess\src\..\include -I..\3rdparty\stout\include -I3rdparty\libapr-1.5.2\src\libapr-1.5.2\include -I3rdparty\libapr-1.5.2\src\libapr-1.5.2-build -I3rdparty\boost-1.65.0\src\boost-1.65.0 -I3rdparty\curl-7.57.0\src\curl-7.57.0\include -I3rdparty\elfio-3.2\src\elfio-3.2 -I3rdparty\glog-da816ea70\src\glog-da816ea70\src\windows -I3rdparty\picojson-1.3.0\src\picojson-1.3.0 -I3rdparty\protobuf-3.5.0\src\protobuf-3.5.0\src -I3rdparty\zlib-1.2.8\src\zlib-1.2.8 -I3rdparty\zlib-1.2.8\src\zlib-1.2.8-build -I3rdparty\http_parser-2.6.2\src\http_parser-2.6.2 -I3rdparty\sasl2-2.1.27rc3\src\sasl2-2.1.27rc3-build\include -I3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\include -I3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\generated -I3rdparty\googletest-1.8.0\src\googletest-1.8.0\googlemock\include -I3rdparty\googletest-1.8.0\src\googletest-1.8.0\googletest\include /DWIN32 /D_WINDOWS /W3 /GR /EHsc /bigobj /vd2 /EHsc -DUNICODE -D_UNICODE /w /MDd /Zi /Ob0 /Od /RTC1 /MTd /Fosrc\tests\CMakeFiles\test-helper.dir\utils.cpp.obj /FdTARGET_COMPILE_PDB /FS C:/Users/andschwa/src/mesos/build/C:/Users/andschwa/src/mesos/src/tests/utils.cpp -resource-dir=c:/Users/andschwa/.emacs.d/cquery/build/debug/lib/LLVM-5.0.1-win64/lib/clang/5.0.1 -Wno-unknown-warning-option -fparse-all-comments -fms-extensions -fms-compatibility -fdelayed-template-parsing -fsyntax-only

I've tried doing what it said, and under clang++.exe directly, it fails because the syntax is cl.exe syntax. (Note: I cannot compile the whole project with clang on Windows, some our dependencies e.g. GoogleTest don't support it, but the Mesos sources themselves compile with clang on other platforms).

Running the same arguments under clang-cl.exe gets me a bit further (after putting double quotes around the paths because the embedded dashes otherwise cause more syntax errors), and finally I got to this error and am stumped:

clang-cl.exe  /TP -working-directory C:/Users/andschwa/src/mesos/build -xc++ -std=c++14 -DBUILD_DIR="C:/Users/andschwa/src/mesos/build" -DCURL_STATICLIB -DGOOGLE_GLOG_DLL_DECL= -DHAVE_LIBZ -DLIBDIR="WARNINGDONOTUSEME" -DLIBSASL_EXPORTS=1 -DNOGDI -DNOMINMAX -DPICOJSON_USE_INT64 -DPKGDATADIR="" -DPKGLIBEXECDIR="WARNINGDONOTUSEME" -DPKGMODULEDIR="WARNIN GDONOTUSEME" -DSBINDIR="WARNINGDONOTUSEME" -DSOURCE_DIR="C:/Users/andschwa/src/mesos" -DTESTLIBEXECDIR="WARNINGDONOTUSEME" -DUSE_CMAKE_BUILD_CONFIG -DUSE_STATIC_LIB -DVERSION="1.6.0" -DZOOKEEPER_VERSI ON="3.4.8" -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_WARNINGS -D_SCL_SECURE_NO_WARNINGS -D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING -D__STDC_FORMAT_MACROS -D__WINDOWS__ -I..\include -Iinclude -I"in clude\mesos" -Isrc -I"..\src" -I"..\3rdparty\libprocess\src\..\include" -I"..\3rdparty\stout\include" -I"3rdparty\libapr-1.5.2\src\libapr-1.5.2\include" -I"3rdparty\libapr-1.5.2\src\libapr-1.5.2-build " -I"3rdparty\boost-1.65.0\src\boost-1.65.0" -I"3rdparty\curl-7.57.0\src\curl-7.57.0\include" -I"3rdparty\elfio-3.2\src\elfio-3.2" -I"3rdparty\glog-da816ea70\src\glog-da816ea70\src\windows" -I"3rdpart y\picojson-1.3.0\src\picojson-1.3.0" -I"3rdparty\protobuf-3.5.0\src\protobuf-3.5.0\src" -I"3rdparty\zlib-1.2.8\src\zlib-1.2.8" -I"3rdparty\zlib-1.2.8\src\zlib-1.2.8-build" -I"3rdparty\http_parser-2.6. 2\src\http_parser-2.6.2" -I"3rdparty\sasl2-2.1.27rc3\src\sasl2-2.1.27rc3-build\include" -I"3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\include" -I"3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\s rc\c\generated" -I"3rdparty\googletest-1.8.0\src\googletest-1.8.0\googlemock\include" -I"3rdparty\googletest-1.8.0\src\googletest-1.8.0\googletest\include" /DWIN32 /D_WINDOWS /W3 /GR /EHsc /bigobj /vd 2 /EHsc -DUNICODE -D_UNICODE /w /MDd /Zi /Ob0 /Od /RTC1 /MTd /Fosrc\tests\CMakeFiles\test-helper.dir\utils.cpp.obj /FdTARGET_COMPILE_PDB /FS "C:/Users/andschwa/src/mesos/src/tests/utils.cpp" -Wno-unkn own-warning-option -fparse-all-comments -fms-extensions -fms-compatibility -fdelayed-template-parsing -fsyntax-only
clang-cl.exe: error: cannot specify '/Fosrc\tests\CMakeFiles\test-helper.dir\utils.cpp.obj' when compiling multiple source files

There is only one source file specified, so the error doesn't make sense to me.

Has anyone gotten cquery to work on Windows with a CMake-generated Ninja-driven MSVC project?

[andyleejordan]

Please ignore the random spaces in the pastes, copying from PowerShell is awful.

[MaskRay]

$ /TP -working-directory C:/Users/andschwa/src/mesos/build -xc++ -std=c++14

This is weird. The compiler driver is missing in the list of args.

https://github.com/cquery-project/cquery/blob/master/src/clang_translation_unit.cc#L88

https://github.com/cquery-project/cquery/blob/master/src/project.cc#L147

Is the compiler driver clang-cl.exe recognized by this function? You may LOG_S(INFO) << "+++" << result.args[0] and view it in the --log-file=/tmp/cq.log log file or std::cerr <<< ... and view it in the lsp-cquery stderr buffer.

[Riatre]

See https://github.com/Valloric/ycmd/pull/789, you may need to have --driver-model=cl in all compiler arguments or in cquery.index.extraClangArguments (this is a VSCode thing, I'm not sure what the equivalent Emacs one is).

[andyleejordan]

Okay, so I got a bit further. Using extraClangArgs I got --driver-mode=cl passed in, which is necessary, but I discovered a few more problems. Thanks for the pointer @Riatre.

Passing extra clang arguments using the Emacs extension:

(customize-set-variable 'cquery-extra-init-params '(:extraClangArguments ("--driver-mode=cl"))

However, @MaskRay is also correct, it is bizarre that the arguments don't have the compile driver itself (e.g. clang, or in my case cl.exe which would be wrong even if it found it). There are a couple problems around this. Since my project is configured with MSVC, not Clang, the compiler is cl.exe, and the options start with / instead of -. This screws with the logic found here. My temporary, Windows-only patch looks like this:

   // ie, compiler schedular such as goma. This allows correct parsing for
   // command lines like "goma clang -c foo".
   std::string::size_type dot;
-  while (i < entry.args.size() && entry.args[i][0] != '-' &&
+  while (i < entry.args.size() &&
+         entry.args[i][0] != '-' &&
+         entry.args[i][0] != '/' && // Windows options: WILL BREAK LINUX.
          // Do not skip over main source filename
          NormalizePathWithTestOptOut(entry.args[i]) != result.filename &&
          // There may be other filenames (e.g. more than one source filenames)
...
           !entry.args[i].compare(dot + 1, 3, "exe")))
     ++i;
   // Include the compiler in the args.
+  bool is_msvc = false;
   if (i > 0)
-    result.args.push_back(entry.args[i - 1]);
+    // Replace `cl.exe` with `clang++.exe --driver-mode=cl`.
+    if (entry.args[i - 1].compare(entry.args[i].size() - 6, 6, "cl.exe")) {
+      is_msvc = true;
+      result.args.push_back("clang++.exe");
+      result.args.push_back("--driver-mode=cl");
+    }
+    else
+      result.args.push_back(entry.args[i - 1]);
   else {
     // TODO Drop this back compatibility
     // Args probably came from a /.cquery file, which likely has just flags.

So far with this I can properly keep all the arguments, and replace cl.exe with clang++.exe --driver-mode=cl (also negates needing to set that in Emacs Lisp). This lets the rest of the cl.exe arguments (well, almost) be understood by clang++.exe.

Another problem is that I have an argument, /permisssive-, that clang-cl does not recognize (but should). For now, I'm adding this to cquery's internal blacklist, but should also upstream this to the Clang project. It's a valid cl.exe argument.

Similarly, cquery assumes the compile driver is clang, not clang in cl-mode. So it adds the arguments -working-directory <build/path>, -xc++, -std=c++14, etc., which now break clang in cl mode. The former actually breaks it entirely because of the way the argument is structured, the latter ones just get ignored. I am currently working on mapping these (-Xclang looks promising).

Finally, the Windows absolute-path logic assumes the paths always use /, never \, which isn't true in my case with cl.exe.

[MaskRay]

Command line parsing is really hard... https://github.com/cquery-project/cquery/issues/236 https://github.com/cquery-project/cquery/issues/377

[andyleejordan]

Indeed. Ah man but I want to get this working so bad since pretty much nothing else works with Mesos well. Especially on Windows.

[MaskRay]

I actually want to remove these -xc -std=c11 -xc++ -std=c++14 flags as they complicate the logic, but I guess someone will complain about that (I incline to disregard the complaint)

The compiler driver detection logic may be replaced by detecting if the first few arguments are executable. I wonder if goma appears in the compile_commands.json "arguments" used for building Chrome.

We may add an option to invoke an external script to give us the processed compile_commands.json contents, not the file on the disk, as something similar to .ycm_extra_conf.py

How does -working-directory break clang-cl?

[MaskRay]

You may also use .cquery which overrides compile_commands.json. Since d5ce5845b2391138f2b4b9f2e25d6b87ac32dbe6 the file can exist in sub directories.

emacs-cquery locates the project root with (locate-dominating-file default-directory ".cquery"), which needs changing for hierarchical .cquery files. https://github.com/cquery-project/emacs-cquery/blob/master/cquery-common.el#L50

[MaskRay]

https://github.com/cquery-project/cquery/wiki/compile_commands.json#stdout-of-an-external-command

Please add the support for Windows

https://github.com/cquery-project/cquery/commit/eadc53abe9af1e28d0a1abba40da7a16864de919

[bstaletic]

I actually want to remove these -xc -std=c11 -xc++ -std=c++14 flags

I am not sure if that's a good idea. Last year I had a project for which I intentionally supplied -xc -std=c89 expecting the compiler to respect it and warn me about anything that got introduced in later standards.

The compiler driver detection logic may be replaced by detecting if the first few arguments are executable.

What about the following:

• clang-cl --driver-mode=gcc
• clang++ --driver-mode=cl

[MaskRay]

I actually want to remove these -xc -std=c11 -xc++ -std=c++14 flags

I am not sure if that's a good idea. Last year I had a project for which I intentionally supplied -xc -std=c89 expecting the compiler to respect it and warn me about anything that got introduced in later standards.

I think we are on the same page :) Before yesterday, cquery adds -std=c11 or -std=c++14 if the project does not specify -std=. I changed that and now it only adds options when .cquery is used, not when compile_commands.json is used. https://github.com/cquery-project/cquery/wiki/Getting-started#cquery

What about the following: clang-cl --driver-mode=gcc clang++ --driver-mode=cl

The "Strip all arguments consisting the compiler command," logic is used to support goma or other compiler scheduler. https://github.com/cquery-project/cquery/blob/master/src/project.cc#L146 I dislike the logic and do not use it in "compilationDatabaseCommand" mode.

[andyleejordan]

@MaskRay

Re: working-directory

How does -working-directory break clang-cl?

clang-cl doesn't recognize the -working-directory flag, as it expects only cl-like flags. So these arguments:

clang++.exe --driver-mode=cl -working-directory C:/Users/andschwa/src/mesos/build

Produce this error:

error: error reading 'C:/Users/andschwa/src/mesos/build'

Re: compiledb

We may add an option to invoke an external script to give us the processed compile_commands.json contents, not the file on the disk

This could be interesting, given this extension.

That said, I'm happy to use Ninja and therefore CMake's CMAKE_EXPORT_COMPILE_COMMANDS to produce a compile_commands.json. It's tricky to integrate clang-cl support, but I think doable. The logic of "if compile driver == cl.exe, use clang++ --driver-mode-cl` seems sound.

But I'll also try a compiledb. It might work better.

[MaskRay]

At the cost of extra 3 lines, e15f57f9c732b1c544e6e7bf2ae8a2729c9eb41a, .cquery for clang-cl is supported.

These should work.

# .cquery
% cat /tmp/d/.cquery
clang-cl
/I
inc

[
    { "arguments": [ "clang-cl", "/Iinc", "a.cc" ], "directory": "/tmp/c", "file": "a.cc" }
]

Please add support for Windows for this commit https://github.com/cquery-project/cquery/commit/eadc53abe9af1e28d0a1abba40da7a16864de919 and use "compilationDatabaseCommand" if you want to support more complicated setting.

[MaskRay]

Welcome to discuss Windows support in https://gitter.im/cquery-project/Lobby

Also see https://github.com/cquery-project/cquery/pull/501

[bstaletic]

clang-cl doesn't recognize the -working-directory flag, as it expects only cl-like flags.

That's not correct. clang-cl /I include_dir_foo -I include_dir_bar -c foo.c -o foo happily produces foo.obj respecting both -I and /I - at least on unix-like systems.

[MaskRay]

clang -working-directory=/tmp/c -c a.cc
clang --driver-mode=cl -working-directory=/tmp/c -c a.cc

I guess cl mode takes -working-directory as an unknown warning option and ignores it.

[andyleejordan]

I guess cl mode takes -working-directory as an unknown warning option and ignores it.

Exactly this.

[andyleejordan]

Please add support for Windows for this commit eadc53a and use "compilationDatabaseCommand" if you want to support more complicated setting.

Sorry, I had to put this on hold for a bit. I'll try this out next. I really want to get this working.

Ironically I need to implement the same semantics here as in Mesos. Should be doing it this week.

[bstaletic]

Note that CL driver support is relatively new and has not seen much use, so bugs are to be expected. Someone should report the -working-directory bug to the llvm devs.

[DaanDeMeyer]

https://bugs.llvm.org/show_bug.cgi?id=36648

[bstaletic]

Back to the original topic of parsing command line, yes it's quite hard and annoying. Since there's always a question which directory are the paths relative to, ycmd has include_paths_relative_to_dir as part of its .ycm_extra_conf.py. So instead of supplying -working-directory to libclang, ycmd uses include_paths_relative_to_dir to make all paths absolute.

@andschwa Ycmd isn't as feature rich, but maybe you'd have less troubles getting it to work with your project.

@MaskRay I hope you won't mind me "promoting" ycmd.

[DaanDeMeyer]

https://github.com/cquery-project/cquery/pull/501 Changes cquery to do the same. Has ycmd had any issues with this approach?

[bstaletic]

Not at all. At least not with the absolute paths part of the code.

Ycmd is more aggressive when it comes to sanitising flags. It will remove all positional parameters that are not include paths. This caused some issues with recognising when /I is a flag and not an absolute path, but I believe we have taken care of CL driver by now.

[jacobdufault]

I hope you won't mind me "promoting" ycmd.

That's fine :)

[andyleejordan]

Thanks @jacobdufault for implementing GetExternalCommandOutput. Work and a vacation got in the way.

[jacobdufault]

Sure :)

[andyleejordan]

@MaskRay With the latest version of cquery (built with bundled Clang 6.0.0 after reading #219), this is now working for me with Mesos on Windows. I am so happy. I can jump to definitions, find references, have syntax errors highlighted, preprocessor guards are greyed, completions are available, the works! I'm sorry I wasn't more helpful. I think where I left off was trying to get cquery built with my own system clang, and then llvm-config wasn't working (and 6.0.0 wasn't bundle yet... or even released), and I had to get back to work. I saw several things go in that I'd (sort of) fixed in a WIP branch I never posted :(

But anyway, thank you all. This is f***ing fantastic.

[andyleejordan]

Oh, and my setup is just using Ninja to generate a compile_commands.json, which with the clang-cl driver mode, is working just fine. I had to make Ninja work on Mesos' Windows build, which was a whole different story... but it's faster, so improvements were found there too.

[jacobdufault]

:), cquery has had some quality-of-life improvements for windows recently (such as automatically discovering system include headers). You may still run into some issues with vscode opening duplicate files with different path casing.