libclang: version 3.8 required?

[gavanderhoorn]

Would other versions work?

[git-afsantos]

I have not tested other versions yet. 3.8 is what is available for the Ubuntu version corresponding to Kinetic (and Indigos as well). I think Mirko is working with 3.8.1, and he only had to make a few deviations in the setup, something to do with sym links, if I recall correctly.

I believe other versions might work, as long as you install the correct Python bindings for the libclang version, and as long as that version does not break anything that bonsai is currently using. For HAROS itself the versions do not really matter.

[gavanderhoorn]

Hm. I've got 3.8 installed on Xenial and running analyse -n but the model extraction doesn't seem to work (ie: loading the report in my browser and going to the Components tab doesn't show anything).

There's also no info on topics, messages or anything under ROS Communications.

There are definitely publishers, subscribers and other interesting bits in the packages that the model extraction should pick up on.

Running with --debug results in a large log file. Anything I could look for that would help diagnosing this?

[git-afsantos]

Hm. I've got 3.8 installed on Xenial and running analyse -n but the model extraction doesn't seem to work (ie: loading the report in my browser and going to the Components tab doesn't show anything).

Nothing at all, or does it only show the white nodes? Do you have any configurations (i.e. apps) defined in the project YAML?

There's also no info on topics, messages or anything under ROS Communications.

I think this block was never properly implemented, perhaps I should check it out just to be sure.

If I recall correctly, the problem was that the user may have many different configurations, and then I would not know how to count the primitives (number of unique ones globally?). This might be one of the things that I left for future redesign, but never got to it.

Running with --debug results in a large log file. Anything I could look for that would help diagnosing this?

Can you find any line that starts with the following?

DEBUG:haros.extractor:Extracting primitives for node

[gavanderhoorn]

Hm. I've got 3.8 installed on Xenial and running analyse -n but the model extraction doesn't seem to work (ie: loading the report in my browser and going to the Components tab doesn't show anything).

Nothing at all, or does it only show the white nodes?

Nothing at all.

The layout of the window is also "strange" (I'll add a screenshot when I get back to my machine).

Do you have any configurations (i.e. apps) defined in the project YAML?

No, no configuration.

I'm trying to see whether I can avoid those as it would make using HAROS as part of a CI run on the ROS buildfarm easier (haros_catkin).

There's also no info on topics, messages or anything under ROS Communications.

I think this block was never properly implemented, perhaps I should check it out just to be sure.

The screenshot in the readme does show these populated (or is that a 'fake'?):

If I recall correctly, the problem was that the user may have many different configurations, and then I would not know how to count the primitives (number of unique ones globally?). This might be one of the things that I left for future redesign, but never got to it.

Running with --debug results in a large log file. Anything I could look for that would help diagnosing this?

Can you find any line that starts with the following?

DEBUG:haros.extractor:Extracting primitives for node

I'll update this when I get back to my machine.

[git-afsantos]

No, no configuration.

That explains it. Nothing under configurations in the YAML implies nothing for -n to work on.

This is something I have been discussing recently, whether it would be worth it to build isolated models of nodes found within the packages, without launch file context.

[gavanderhoorn]

There could still be context, right? Packages in the same repository / workspace could contain launch files depending on nodes from those packages.

If I understand the context part correctly: wouldn't every launch file essentially provide context?

[git-afsantos]

If I understand the context part correctly: wouldn't every launch file essentially provide context?

In theory, yes. In practice, there are many launch files whose purpose is to be included as part of another top-level launch file. Instead of guessing which is which, I was relying on the user to specify which groups of launch files make a complete configuration.

There are a few options here, e.g.

1. make models for the individual nodes;
2. make models for each launch file;
3. try to filter only the top-level launch files and make models for those.

[git-afsantos]

@gavanderhoorn I do not think any of these options is a simple quick-fix, so I am not sure when will I implement it, but do you have an opinion on which of these options make the most sense?

[git-afsantos]

Back to the original question: I have just set up a ROS Melodic machine (Ubuntu 18.04.1) where libclang-3.8-dev is not available. The available options for Ubuntu 18 are:

• libclang-3.9-dev (3.9.1)
• libclang-4.0-dev (4.0.1)
• libclang-5.0-dev (5.0.1)
• libclang-6.0-dev (6.0.0)
• libclang-7-dev

So far, I have tried 3.9 and 4.0 and both seem to work for the basic examples of https://github.com/git-afsantos/haros_tutorials. There is always the chance that something might have broken backward compatibility (especially for 4.0 and onward), but only more complex examples will reveal that.

There is only one requirement for these versions to work, which is providing the pointers to the correct version in the ~/.haros/configs.yaml file. Here is the example I am using for 4.0.

%YAML 1.1
---
workspace: '/home/andre/ros/ws'
cpp:
    parser_lib: '/usr/lib/llvm-4.0/lib'
    std_includes: '/usr/lib/llvm-4.0/lib/clang/4.0.1/include'
    compile_db: '/home/andre/ros/ws/build'

I will try the remaining versions soon and will provide an update then.

[gavanderhoorn]

@git-afsantos: I haven't checked the code, but are the absolute paths necessary?

[git-afsantos]

Not sure. I would have to try passing a relative path to clang and see if it works. I will give it a go when I get to this again.

[gavanderhoorn]

This seems to suggest it isn't.

On my system libclang.so does exist, but it's in /usr/lib/llvm-3.6/lib which doesn't appear to be on the default search path (libclang-3.6.so.1 does appear in ldconfig -p output though).

[git-afsantos]

@gavanderhoorn I did try to pass a relative path to cpp:parser_lib (even copied the libclang.so to another directory) and it worked just the same.

Also, I have just confirmed that 5.0.1 and 6.0.0 do work with the example above (as long as you change the configs.yaml accordingly and the Python clang package matches your compiler version).

libclang-7-dev does not seem to work, as the corresponding Python version is not released in PyPi yet. https://pypi.org/project/clang/#history

[gavanderhoorn]

And is this with or without setting the library path?

If it's without, it would suggest that there is basically no special configuration needed to use libclang with Haros (which I sort-of would've expected).

[git-afsantos]

And is this with or without setting the library path?

Do you mean setting the LD_LIBRARY_PATH in the environment? If so, it is without. All I did, on a fresh VM, was set up a virtualenv with everything I needed for Haros, and then:

sudo apt-get install clang-3.9 libclang-3.9-dev
pip install -Iv clang==3.9
catkin_make --force-cmake -DCMAKE_CXX_COMPILER= [...]

(update ~/.haros/configs.yaml)

python haros-runner.py [...]
sudo apt-get purge clang-3.9 libclang-3.9-dev
sudo apt-get autoremove
pip uninstall clang
sudo apt-get install clang-4.0 libclang-4.0-dev
pip install -Iv clang==4.0

(repeat)

there is basically no special configuration needed to use libclang with Haros

True. It is just that at some point Haros (bonsai, actually) has to call clang.Config.set_library_path and pass it something, so it has to know where to look (see https://github.com/llvm-mirror/clang/blob/cab7f1f7bc141ac88d15030088b311bd450b2c94/bindings/python/clang/cindex.py#L4104-L4105).

What I could do is update this call so that it tries to search for other versions besides 3.8 and go with whatever matches first.

[git-afsantos]

The original question seems to be answered (other versions work), so I am closing this issue. Let me know if there is still something pending regarding this topic.