Sarcasm / compdb

The compilation database Swiss army knife
MIT License
295 stars 23 forks source link

compdb doubly-extra escapes double quotes in compilation command #19

Closed unphased closed 2 years ago

unphased commented 4 years ago

I am mainly using compdb together with Bear, so this may be specific to the processing of "arguments" formatted input. But if a compilation flag such as -DVAR="STRING" is sent in, where it appears with this JSON compliant syntax:

<...>
{
  "arguments": [
    <...>
    "-DVAR=\"STRING\"",
    <...>
  ],
  <...>
}

The result is

<...>
"command": "c++ <...> -DVAR=\\\"STRING\\\" <...> ",
<...>

So far the editors & tools i'd configured, VSCode's c++ plugin, and ccls, were fine with this (even though I noticed the problem when setting up my scripting), but now my scripting when invoking clang fails with

<command line>:9:27: error: missing terminating '"' character [-Werror,-Winvalid-pp-token]

because it is attempting to generate invalid C++ code that includes the extra (incorrectly python-escaped) backslash...

Sarcasm commented 3 years ago

Copy-pasting my answer from here:

I think compdb is correct, it could just be an issue with incorrect shell command parsing from the tool you use. The escaping performed by compdb is on purpose, because JSON compilation database command is a shell command in a JSON string. compdb outputs entries with the command key, not arguments, so it's normal that there is a difference in escaping between compdb and bear. If the tool correctly parses command using something like Python shlex.split() it should be ok.

Confusingly, I think the official JSON Compilation Database documentation example is wrong:

Example:

[
 { "directory": "/home/user/llvm/build",
   "command": "/usr/bin/clang++ -Irelative -DSOMEDEF=\"With spaces, quotes and \\-es.\" -c -o file.o file.cc",
   "file": "file.cc" },
 ...
]

-- https://clang.llvm.org/docs/JSONCompilationDatabase.html

But from what I remember, tools producing command where mostly correct, and libclang JSON parsing was too.

/usr/bin/clang++ -Irelative -DSOMEDEF="With spaces, quotes and \-es." -c -o file.o file.cc

Example of parsing issue with and without proper escaping:

$ tail -n +1 /tmp/print_args.py /tmp/command.json 
==> /tmp/print_args.py <==
#!/usr/bin/env python
import json
import shlex
import sys

with open(sys.argv[1]) as fp:
    data = json.load(fp)

for i, entry in enumerate(data):
    if i != 0:
        print("")
    arguments = shlex.split(entry["command"], posix=True)
    for argument in arguments:
        print(argument)

==> /tmp/command.json <==
[
 { "directory": "/home/user/llvm/build",
   "command": "clang++ -DGIT_VERSION=\"9c90bc5c\"",
   "file": "foo.cc"
 },
 { "directory": "/home/user/llvm/build",
   "command": "clang++ -DGIT_VERSION=\\\"9c90bc5c\\\"",
   "file": "bar.cc"
 }
]
$ python /tmp/print_args.py /tmp/command.json
clang++
-DGIT_VERSION=9c90bc5c

clang++
-DGIT_VERSION="9c90bc5c"

Ideally, compdb should output arguments instead of command, which is less confusing to use.

zancas commented 2 years ago

It seems like @rizsotto has agreed with @Sarcasm here:

https://github.com/rizsotto/Bear/issues/204#issuecomment-903601973

Should this issue should be closed?

Sarcasm commented 2 years ago

Yes, closing. I would accept contributions that use arguments instead of command, I don't plan working on it myself for the time being.