search

Sarcasm / compdb

The compilation database Swiss army knife
MIT License
292 stars 23 forks source link

Cannot handle windows-style path and MSVC options #24

Closed ChihHao-Su closed 2 years ago

ChihHao-Su commented 2 years ago

The origin compile_commands.json:

[
{
  "directory": "D:/PROJ/CPP/pointcloud_remesh/BUILD",
  "command": "\"C:\\DEV_TOOL\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.28.29333\\bin\\Hostx64\\x64\\cl.exe\"  /nologo /TP -DBOOST_ALL_NO_LIB -DBOOST_ALL_NO_LIB-DBOOST_ALL_NO_LIB -DBOOST_ATOMIC_NO_LIB -DBOOST_DATE_TIME_NO_LIB -DBOOST_FILESYSTEM_NO_LIB -DBOOST_IOSTREAMS_NO_LIB -DBOOST_SERIALIZATION_NO_LIB -DBOOST_SYSTEM_NO_LIB -DCore_EXPORTS -DKISSFFT_DLL_IMPORT=1 -D__SSE2__ -D__SSE3__ -D__SSE4_1__ -D__SSE4_2__ -D__SSE__ -D__SSSE3__ -Dkiss_fft_scalar=double -I\"D:\\DEV_TOOL\\PCL 1.12.1\\3rdParty\\Eigen\\eigen3\" -I\"D:\\DEV_TOOL\\PCL 1.12.1\\3rdParty\\Boost\\include\\boost-1_78\" -I\"D:\\DEV_TOOL\\PCL 1.12.1\\include\\pcl-1.12\" -I\"C:\\Program Files\\OpenNI2\\Include\" -I\"D:\\DEV_TOOL\\PCL 1.12.1\\3rdParty\\VTK\\include\\vtk-9.1\" -I\"D:\\DEV_TOOL\\PCL 1.12.1\\3rdParty\\VTK\\include\\vtk-9.1\\vtkfreetype\\include\" /DWIN32 /D_WINDOWS /GR /EHsc /Zi /Ob0 /Od /RTC1 -MDd /arch:AVX2 -std:c++latest /FoCore\\CMakeFiles\\Core.dir\\core.cxx.obj /FdTARGET_COMPILE_PDB /FS -c D:\\PROJ\\CPP\\pointcloud_remesh\\Core\\core.cxx",
  "file": "D:/PROJ/CPP/pointcloud_remesh/Core/core.cxx"
}
]

Run with compdb -p ./ list :

[
{
  "directory": "D:/PROJ/CPP/pointcloud_remesh/BUILD",
  "command": "\"C:\\\\DEV_TOOL\\\\Microsoft Visual Studio\\\\2019\\\\Community\\\\VC\\\\Tools\\\\MSVC\\\\14.28.29333\\\\bin\\\\Hostx64\\\\x64\\\\cl.exe\" /nologo /TP -DBOOST_ALL_NO_LIB -DBOOST_ALL_NO_LIB-DBOOST_ALL_NO_LIB -DBOOST_ATOMIC_NO_LIB -DBOOST_DATE_TIME_NO_LIB -DBOOST_FILESYSTEM_NO_LIB -DBOOST_IOSTREAMS_NO_LIB -DBOOST_SERIALIZATION_NO_LIB -DBOOST_SYSTEM_NO_LIB -DCore_EXPORTS -DKISSFFT_DLL_IMPORT=1 -D__SSE2__ -D__SSE3__ -D__SSE4_1__ -D__SSE4_2__ -D__SSE__ -D__SSSE3__ -Dkiss_fft_scalar=double \"-ID:\\\\DEV_TOOL\\\\PCL 1.12.1\\\\3rdParty\\\\Eigen\\\\eigen3\" \"-ID:\\\\DEV_TOOL\\\\PCL 1.12.1\\\\3rdParty\\\\Boost\\\\include\\\\boost-1_78\" \"-ID:\\\\DEV_TOOL\\\\PCL 1.12.1\\\\include\\\\pcl-1.12\" \"-IC:\\\\Program Files\\\\OpenNI2\\\\Include\" \"-ID:\\\\DEV_TOOL\\\\PCL 1.12.1\\\\3rdParty\\\\VTK\\\\include\\\\vtk-9.1\" \"-ID:\\\\DEV_TOOL\\\\PCL 1.12.1\\\\3rdParty\\\\VTK\\\\include\\\\vtk-9.1\\\\vtkfreetype\\\\include\" /DWIN32 /D_WINDOWS /GR /EHsc /Zi /Ob0 /Od /RTC1 -MDd /arch:AVX2 -std:c++latest /FoCoreCMakeFilesCore.dircore.cxx.obj /FdTARGET_COMPILE_PDB /FS -c D:PROJCPPpointcloud_remeshCorecore.cxx",
  "file": "D:/PROJ/CPP/pointcloud_remesh/Core/core.cxx"
},

{
  "directory": "D:/PROJ/CPP/pointcloud_remesh/BUILD",
  "command": "\"C:\\\\DEV_TOOL\\\\Microsoft Visual Studio\\\\2019\\\\Community\\\\VC\\\\Tools\\\\MSVC\\\\14.28.29333\\\\bin\\\\Hostx64\\\\x64\\\\cl.exe\" /nologo /TP -DBOOST_ALL_NO_LIB -DBOOST_ALL_NO_LIB-DBOOST_ALL_NO_LIB -DBOOST_ATOMIC_NO_LIB -DBOOST_DATE_TIME_NO_LIB -DBOOST_FILESYSTEM_NO_LIB -DBOOST_IOSTREAMS_NO_LIB -DBOOST_SERIALIZATION_NO_LIB -DBOOST_SYSTEM_NO_LIB -DCore_EXPORTS -DKISSFFT_DLL_IMPORT=1 -D__SSE2__ -D__SSE3__ -D__SSE4_1__ -D__SSE4_2__ -D__SSE__ -D__SSSE3__ -Dkiss_fft_scalar=double \"-ID:\\\\DEV_TOOL\\\\PCL 1.12.1\\\\3rdParty\\\\Eigen\\\\eigen3\" \"-ID:\\\\DEV_TOOL\\\\PCL 1.12.1\\\\3rdParty\\\\Boost\\\\include\\\\boost-1_78\" \"-ID:\\\\DEV_TOOL\\\\PCL 1.12.1\\\\include\\\\pcl-1.12\" \"-IC:\\\\Program Files\\\\OpenNI2\\\\Include\" \"-ID:\\\\DEV_TOOL\\\\PCL 1.12.1\\\\3rdParty\\\\VTK\\\\include\\\\vtk-9.1\" \"-ID:\\\\DEV_TOOL\\\\PCL 1.12.1\\\\3rdParty\\\\VTK\\\\include\\\\vtk-9.1\\\\vtkfreetype\\\\include\" /DWIN32 /D_WINDOWS /GR /EHsc /Zi /Ob0 /Od /RTC1 -MDd /arch:AVX2 -std:c++latest /FoCore\\CMakeFiles\\Core.dir\\core.cxx.obj /FdTARGET_COMPILE_PDB /FS D:\\PROJ\\CPP\\pointcloud_remesh\\Core\\core.cxx -c D:\\\\PROJ\\\\CPP\\\\pointcloud_remesh\\\\Core\\\\core.hpp",
  "file": "D:\\PROJ\\CPP\\pointcloud_remesh\\Core\\core.hpp"
}
]
  1. Some of \\ become \\\\, some disappeared.
  2. Some \" moved before the option. (e.g.: -I\"D:\\DEV_TOOL\\PCL 1.12.1\\3rdParty\\VTK\\include\\vtk-9.1\" become \"-ID:\\\\DEV_TOOL\\\\PCL 1.12.1\\\\3rdParty\\\\VTK\\\\include\\\\vtk-9.1\")
  3. Maybe something other. The output is confused.
Sarcasm commented 2 years ago

Do you have any issue / error with the compdb output? We do not try to preserve the command line specific formatting, just the semantics.

  1. Although ugly, the quadruple backslashes are more correct than the double ones, at least on Linux. This is because the command field is a command line, and we need to shell-escape \ -> \\ it, then json-escape it \\ -> \\\\.
  2. The semantic should be preserved.
ChihHao-Su commented 2 years ago

The real problem is, compdb will eat up some \. This often happens in path in options and path of source file (and maybe somewhere other). e.g.: /FoCMakeFiles\main.dir\main.cxx.obj will become /FoCMakeFilesmain.dirmain.cxx.obj , D:\PROJ\CPP\HelloWorld\main.cxx will become D:PROJCPPHelloWorldmain.cxx . Therefore clangd cannot parse my c++ code with the compile_commands.json compdb generated.

On 2022/1/18 20:18, Guillaume Papin wrote:

Do you have any issue / error with the |compdb| output? We do not try to preserve the command line specific formatting, just the semantics.

Although ugly, the quadruple backslashes are more correct than the double ones, at least on Linux. This is because the |command| field is a command line, and we need to shell-escape || -> |\| it, then json-escape it |\| -> |\\|.

— Reply to this email directly, view it on GitHub https://github.com/Sarcasm/compdb/issues/24#issuecomment-1015356602, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFKCVUXSGX6JCNZAWFGDMVLUWVLCDANCNFSM5MGRZDEA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

ChihHao-Su commented 2 years ago

There should be another issue that processing compile_commands.json while using MSVC compiler. After I repair that problem manually (adding missing \\) :

[
{
  "directory": "D:/PROJ/CPP/HelloWorld/BUILD",
  "command": "\"C:\\\\DEV_TOOL\\\\Microsoft Visual Studio\\\\2019\\\\Community\\\\VC\\\\Tools\\\\MSVC\\\\14.28.29333\\\\bin\\\\Hostx64\\\\x64\\\\cl.exe\" /nologo /TP /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MDd /Zi /Ob0 /Od /RTC1 -MDd /FoCMakeFiles\\main.dir\\core.cxx.obj /FdTARGET_COMPILE_PDB /FS -c D:\\PROJ\\CPP\\HelloWorld\\core.cxx",
  "file": "D:/PROJ/CPP/HelloWorld/core.cxx"
},
{
  "directory": "D:/PROJ/CPP/HelloWorld/BUILD",
  "command": "\"C:\\\\DEV_TOOL\\\\Microsoft Visual Studio\\\\2019\\\\Community\\\\VC\\\\Tools\\\\MSVC\\\\14.28.29333\\\\bin\\\\Hostx64\\\\x64\\\\cl.exe\" /nologo /TP /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MDd /Zi /Ob0 /Od /RTC1 -MDd /FoCMakeFiles\\main.dir\\main.cxx.obj /FdTARGET_COMPILE_PDB /FS -c D:\\PROJ\\CPP\\HelloWorld\\main.cxx",
  "file": "D:/PROJ/CPP/HelloWorld/main.cxx"
},
{
  "directory": "D:/PROJ/CPP/HelloWorld/BUILD",
  "command": "\"C:\\\\DEV_TOOL\\\\Microsoft Visual Studio\\\\2019\\\\Community\\\\VC\\\\Tools\\\\MSVC\\\\14.28.29333\\\\bin\\\\Hostx64\\\\x64\\\\cl.exe\" /nologo /TP /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MDd /Zi /Ob0 /Od /RTC1 -MDd /FoCMakeFiles\\main.dir\\core.cxx.obj /FdTARGET_COMPILE_PDB /FS D:\\PROJ\\CPP\\HelloWorld\\core.cxx -c D:\\\\PROJ\\\\CPP\\\\HelloWorld\\\\core.hpp",
  "file": "D:\\PROJ\\CPP\\HelloWorld\\core.hpp"
}
]

(This is the origin output of compdb: )

[
{
  "directory": "D:/PROJ/CPP/HelloWorld/BUILD",
  "command": "\"C:\\\\DEV_TOOL\\\\Microsoft Visual Studio\\\\2019\\\\Community\\\\VC\\\\Tools\\\\MSVC\\\\14.28.29333\\\\bin\\\\Hostx64\\\\x64\\\\cl.exe\" /nologo /TP /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MDd /Zi /Ob0 /Od /RTC1 -MDd /FoCMakeFilesmain.dircore.cxx.obj /FdTARGET_COMPILE_PDB /FS -c D:PROJCPPHelloWorldcore.cxx",
  "file": "D:/PROJ/CPP/HelloWorld/core.cxx"
},
{
  "directory": "D:/PROJ/CPP/HelloWorld/BUILD",
  "command": "\"C:\\\\DEV_TOOL\\\\Microsoft Visual Studio\\\\2019\\\\Community\\\\VC\\\\Tools\\\\MSVC\\\\14.28.29333\\\\bin\\\\Hostx64\\\\x64\\\\cl.exe\" /nologo /TP /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MDd /Zi /Ob0 /Od /RTC1 -MDd /FoCMakeFilesmain.dirmain.cxx.obj /FdTARGET_COMPILE_PDB /FS -c D:PROJCPPHelloWorldmain.cxx",
  "file": "D:/PROJ/CPP/HelloWorld/main.cxx"
},
{
  "directory": "D:/PROJ/CPP/HelloWorld/BUILD",
  "command": "\"C:\\\\DEV_TOOL\\\\Microsoft Visual Studio\\\\2019\\\\Community\\\\VC\\\\Tools\\\\MSVC\\\\14.28.29333\\\\bin\\\\Hostx64\\\\x64\\\\cl.exe\" /nologo /TP /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MDd /Zi /Ob0 /Od /RTC1 -MDd /FoCMakeFilesmain.dircore.cxx.obj /FdTARGET_COMPILE_PDB /FS D:PROJCPPHelloWorldcore.cxx -c D:\\\\PROJ\\\\CPP\\\\HelloWorld\\\\core.hpp",
  "file": "D:\\PROJ\\CPP\\HelloWorld\\core.hpp"
}
]

clangd shows me Cannot specify 'FoCMakeFiles\main.dir\core.cxx.obj' when compiling multiple source files in core.hpp.

Sarcasm commented 2 years ago

I think the issue comes from the original command, where you do not have the proper json + shell escapes, i.e. \\\\ instead of \\.

ChihHao-Su commented 2 years ago

The original compile_command.json is generated by cmake with Ninja generator.

ChihHao-Su commented 2 years ago

Preprocessing the input compile_command.json should be a workaround.

Sarcasm commented 2 years ago

Ok, so maybe it's Windows shell parsing rules that differs significantly from POSIX. I would be willing to accept PR that fixes this but don't personally plan to work on it.

Thank you for reporting the issue.