[cmake] Adding support for PCH and Unity builds (using Cotire/CMake 3.16)

[nh2]

Have a look at https://github.com/sakra/cotire, it can speed up the build of a CMake project with low effort (by using precompiled headers).

When compiling an example like the extract_indices tutorial, precompiled headers brought down my compile time from 6s to 1.8.

For this example, with g++ -H I found that it includes over 3900 header files. A precompiled header reduces that to 1.

Note that cotire doesn't seem to be able to share precompiled headers across files so far, so it might not be able to help when doing just a single full build once, but it seems like it can dramatically improve build speeds when developing.

The speedup [doesn't seem to work with clang] right now though.

[taketwo]

Thanks for sharing this, I didn't know about cotire, but it looks rather promising. I am not very proficient with cmake and the ins and outs of PCL build system, so not sure how realistic it is to integrate cotire. Hopefully someone more knowledgeable will comment on this.

[nizar-sallem]

It is not the right place for it, we us ccache which allows for the same speed up but doesn't require to modify the build system. I prefer this kind of solution to cotire since it replaces the compiler. It would be interesting to ask ITK to include cotire in cmake rather than ask us to support it. That is my opinion.

[nh2]

@nizar-sallem Unfortunately ccache doesn't help speeding up the include cost of massive C++ template libraries.

[stale[bot]]

Marking this as stale due to 30 days of inactivity. It will be closed in 7 days if no further activity occurs.

[nh2]

Just to be very clear, a stale bot is a very poor idea and you will lose contributors this way. PCL has lots of long-standing problems, and they won't go away by themselves after 30 days.

Most people, including me, are also not going to come here every 30 days to "renew" their issues. If every project did this, as a contributor to many projects, I'd have to renew tens of issues every day.

[kunaltyagi]

Most people are also not going to come here every 30 days to "renew" their issues

We know that. We'd be re-labelling issues as well fine-tuning the stale-bot config to ensure this doesn't happen.

Eg: We're not excluding issues with kind: todo right now, because, ... well, it didn't cross my mind when I updated the configuration to find badly labelled issues.

kind: todo is simply backlog for the maintainers and is a no-brainer to add to stalebot. kind: proposal on the other hand is more difficult, because maintainers have only a limited time, and I like the idea for the topic to get either a Aye/Nay vote without a month of inactivity. Since the maintainers have only a limited bandwidth, community contributions and PR on these issues labelled with kind: proposal would be amazing.

[kunaltyagi]

We can get the discussion restarted, esp wrt:

• what would a PR that resolves this look like?
• how to transition from Cotire to inbuilt PCH and unity build support in CMake since 3.16 (Available in 20.04, can be used from 2023 in PCL if they don't upgrade CMake in 18.04)

[nh2]

From my experience with Cotire and CMake's builtin support for PCH and unity builds:

• In general, unity builds is the easier of the two techniques to try, it needs less configuration and manual work.
• Cotire generated PCH headers automatically, in CMake one has to craft them by hand. That is fine but requires somebody with good overview across the breadth of PCL to compose the PCH (figuring out e.g. which includes are heavy and common everywhere).
• Very useful is to measure the time spent in C-Pre-Processing (header includes) to get an estimate for how much PCH will be able to help. I've had one project where I introduced PCH and it did not lead to much speedup because, contrary to previous expectations, it was not header includes that were slow on that project, but C++ template instantiations, which is not sped up by PCH. (However, the initial measurement I did in the issue description suggests that for PCL, PCH can help up to 5x for that PCL executable).

[kunaltyagi]

You seem to imply that you've got a version of PCH-enabled PCL. Did you manage to measure the improvement due to Unity build too?

[nh2]

You seem to imply that you've got a version of PCH-enabled PCL. Did you manage to measure the improvement due to Unity build too?

Unfortunately I don't have that. For the experiment in the issue description in 2014

When compiling an example like the extract_indices tutorial, precompiled headers brought down my compile time from 6s to 1.8.

I had manually compiled a precompiled header for the extract_indices program and measured with that.

[kunaltyagi]

Personally speaking, I liked this idea, and tried to experiment, but I don't have CMake skills to achieve this. Could you take a swing at it?

[nh2]

I don't think I qualify for

requires somebody with good overview across the breadth of PCL to compose the PCH

as I've mainly worked with KinFu.

I also don't currently have any work involving PCL any more, but I could take a shot at unity builds next time I do (I cannot predict when that will be though).

[kunaltyagi]

I don't think a lot of people do. If you start from GPU and CUDA modules, others can copy from it.

I cannot predict when that will be though

No issues. Putting this on back-burner