search

catkin / catkin_tools

Command line tools for working with catkin
Apache License 2.0
165 stars 146 forks source link

generating compile_commands.json for workspace #551

Open rhklite opened 5 years ago

rhklite commented 5 years ago

System Info

Build / Run Issue

Expected Behavior

Actual Behavior

Is there a way to properly generate a compile_commands.json for the entire catkin workspace using catkin build?

python script here:

import os
import sys

workspace_path = os.popen('catkin locate --workspace $(pwd)').read().rstrip()

if not workspace_path:
    sys.exit("\033[1;31mNo workspace found in the current directory\033[0m")

ccjson_path = workspace_path+'/compile_commands.json'
target_ccjson_path = workspace_path+'/src'+'/compile_commands.json'

for i in [ccjson_path, target_ccjson_path]:
    if os.path.isfile(i):
        os.system('rm %s' % i)
        print("removed previous compile_commands.json located at %s" % i)

# create a dictionary with file names as keys
# and for each file name the paths where they
# were found
file_paths = {}
for root, dirs, files in os.walk(workspace_path):
    for f in files:
        if f.endswith('compile_commands.json'):
            if f not in file_paths:
                file_paths[f] = []
            file_paths[f].append(root)

# for each file in the dictionary, concatenate
# the content of the files in each directory
# and write the merged content into a file
# with the same name at the top directory
for f, paths in file_paths.items():
    txt = []
    with open(os.path.join(paths[0], f)) as f2:
        txt.append('['+f2.read()[1:-2]+',')
    for p in paths[1:-1]:
        with open(os.path.join(p, f)) as f2:
            txt.append(f2.read()[1:-2]+',')
    with open(os.path.join(paths[-1], f)) as f2:
        txt.append(f2.read()[1:-1]+']')
    with open(f, 'w') as f3:
        f3.write(''.join(txt))

if not os.path.isfile(target_ccjson_path):
    os.system('mv ./compile_commands.json %s' % (target_ccjson_path))

print("\033[1;32mFinished writing compile_commands.json\033[0m")

Steps to Reproduce the Issue

cd ~/catkin_ws
catkin clean
catkin build -DCMAKE_EXPORT_COMPILE_COMMANDS=1

Edit: Made the python script work anywhere in the catkin workspace

rhklite commented 5 years ago

there was a bug in my work around python script, i made edits to the script and got it working.

Is there a better way to do this? to generate a compile_commands.json without needing a pythonc script?

ndepal commented 5 years ago

I agree, it would be great if catkin build -DCMAKE_EXPORT_COMPILE_COMMANDS=1 did this.

Based on your python script I made my own shell script that does the same:

#!/bin/sh

cd `catkin locate --workspace $(pwd)`

concatenated="build/compile_commands.json"

echo "[" > $concatenated

first=1
for d in build/*
do
    f="$d/compile_commands.json"

    if test -f "$f"; then
        if [ $first -eq 0 ]; then
            echo "," >> $concatenated
        fi

        cat $f | sed '1d;$d' >> $concatenated
    fi

    first=0
done

echo "]" >> $concatenated
kgreenek commented 4 years ago

If you happen to use YouCompleteMe with vim, you can use this plugin so you don't have to generate the merged compile_commands.json file. Just run catkin config --cmake-args='-DCMAKE_EXPORT_COMPILE_COMMANDS=ON', install the plugin to vim, and everything will just work when you're inside the catkin workspace. Alternatively, you can place the .ycm_extra_conf.py file form that repo in the root of your catkin workspace.

https://github.com/kgreenek/vim-ros-ycm

LeroyR commented 4 years ago

It generates multiple compile commands instead of one merged file as you are working with multiple projects inside. Merging the compile commands sounds like breaking the separation between multiple projects.

So while i understand why you/we/i would want this, it does not sound like something catkin tools should ever do.

edit: catkin_make works since it mushes everything together resulting in packages that seem to work but break if used without the other parts in your workspace (or if build with catkin tools).

AlexisTM commented 4 years ago

@ndepal @rhklite

My way to merge all compile_commands.json together, using the jq tool to simplify integration with other tools.

sudo apt install jq

jq -s 'map(.[])' build_folder/**/compile_commands.json > compile_commands.json

@LeroyR I agree, catkin_tools should keep compile commands split as it is a split workflow.

hgaiser commented 4 years ago

I am doing something similar. It isn't pretty, but it doesn't require an extra tool:

printf '[' > compile_commands.json
find ./build_isolated -type f -name 'compile_commands.json' -exec sh -c "cat {} | tail -n+2 | head -n-1 && printf ','" >> compile_commands.json \;
sed -i '$s/.$//' compile_commands.json
printf '\n]\n' >> compile_commands.json
stertingen commented 3 years ago

Interestingly, colcon does this: https://github.com/colcon/colcon-cmake/pull/69

zoenglinghou commented 3 years ago

I used @AlexisTM 's comment and come up with a script

wafts95 commented 2 years ago

I came across another use case for having a common compile_commands.json file, and that is to enable the use of the SciTools Understand IDE (link) with ROS projects.