Transition OF to CMake?

[bilderbuchi]

We could transition the OF build system to use CMake. This could have several benefits:

• Ease library compilation
• Create project files for IDEs we use. The PG could also use this
• Less need to keep project files compiling by tweaking manually, as the cmake configuration is the canonical source for all of them)

@procedural has a cmake-based fork of OF going here. Some previous discussion in this comment ff.

Discuss!

[arturoc]

There's a couple of problems with cmake:

• it's not bidirectional: ie: if i add an addon to an already create xcode project using the xcode plugin or even a library manually to the project and then i try to update the project through cmake, it's going to overwrite those change.
• the current system works perfectly ok, our current makefile system for example handle dependencies really well, i've never have to clean a project, which seems to be happening with cmake according to: https://github.com/openframeworks/openFrameworks/issues/4177 In general we have to know what would this add that we don't have already, for example bilderbuchi, what do you mean with:
  • ease library compilation? do you mean the external dependencies? we already have apothecary which could probably be moved to cmake too? would that make things easy? anyone using linux or osx knows a bit of bash but people would need to learn cmake
  • Create project files for IDEs we use. The PG could also use this: The PG already does this and in some cases, ie: linux is even bidirectional already, in other cases we could make it, if we use cmake i doubt we could make it bidirectional
  • Less need to keep project files compiling by tweaking manually, as the cmake configuration is the canonical source for all of them)? do you mean by OF developers? not sure what this means

For me the only pros are that qt creator can use cmake directly which could allow us to get rid of codeblocks and that cmake is becoming the c++ standard lately so it could make some things like creating deb packages slightly easier but not even sure about that

[arturoc]

also how's support for android for example?

[bilderbuchi]

ease library compilation? do you mean the external dependencies? we already have apothecary which could probably be moved to cmake too? would that make things easy? anyone using linux or osx knows a bit of bash but people would need to learn cmake

apparently, there are problems with compiling some libraries, which can be solved with cmake

The PG already does this and in some cases, ie: linux is even bidirectional already, in other cases we could make it, if we use cmake i doubt we could make it bidirectional

yes the PG does this, but from what I have seen so far there's lots of manual work: e.g. we manually parse xml for the xcode files (with lots of breakage and headache for @ofZach iirc), and copy file templates around. cmake could generate those from one config description, in an automated fashion

do you mean by OF developers? not sure what this means

yes, see above

qt creator can use cmake directly which could allow us to get rid of codeblocks

why would you get rid of codeblocks? the nice thing is that with cmake we can generate project file for every supported IDE we (or the users) want - our currently supported set (makefiles, XCode, VS, CB, Eclipse), but also others like Codelite (I think @tgfrerer or someone else requested this previously) or sublimetext 2. And this is (as far as I can see) without keeping fragile templates or XML parsers or parsing/munging code around to maintain.

About bidirectionality I have no answer. @procedural ? Android: same, no idea currently, but people seem to do it on a quick glance.

[ghost]

CMake is not bidirectional, indeed, but as most of high level things are: you don't expect to tweak the assembly and see C code updated automatically... :)

The PG already does this and in some cases, ie: linux is even bidirectional already, in other cases we could make it

That's cool, really. I just don't see [for me personally!] how it will be useful since I don't use makefiles at all (see below).

And this is (as far as I can see) without keeping fragile templates or XML parsers or parsing/munging code around to maintain.

Exactly. All I can see for a past year or two in openFrameworks is, like [speculating], 60% of maintainer's time devoted for developing code and like 40% for maintaining makefiles / PG and solving bugs related to them. Half of the forum is filled not with questions like how to move objects in space or things like that related to oF code but with error reports of undefined symbols and compile time errors.

Aside from bidirectionality, the bigger problem I see with maintaining project files like oF does is their asynchronous updates. Don't forget to update VS 2013 projects, don't forget to update VS 2015 projects, don't forget to update Eclipse projects, don't forget to update Xcode projects, don't forget to update Makefiles... Oops, we forgot to update Code::Blocks projects. In ofnode/of I update CMakeLists.txt once and I get the changes in Qt Creator on Linux, I reboot to Windows and I get update in Visual Studio 2013, but oh no, VS2013 doesn't support some feature, let me switch to VS2015 cause it's updated too, but wait, someone brought macbook and he wants to compile my application on it, guess what, just generate updated project file for that as well. Whoa, there's new IDE now, CLion, I'm sure it should support Makefiles, right?...

GNU people are the worst what could happen to open source community. Their licenses are weird, their code is unreadable, their tools are slow. Makefiles are slow. Like, super slow. You would think that your project compiles slow because it's C++ and maybe compilers aren't that good?.. No. No, they're great. They can compile C++ code without much of templates in less than a second.

“How can this be taking so long,” I’d wonder, “there can’t be that much work to do.”

[bilderbuchi]

You know, I really liked your explanation, you put it better than I could. But this part was really unnecessary:

GNU people are the worst what could happen to open source community. Their licenses are weird, their code is unreadable, their tools are slow.

:-/

[ghost]

@bilderbuchi https://youtu.be/NURiiQatBXA?t=2m58s

[arturoc]

apparently, there are problems with compiling some libraries, which can be solved with cmake

i don't think that's the case, constantine has been putting most time in this and he's been using cmake, i'd say that the same amount of time fixing our current system would have fixed the issue there too not that cmake makes it easier to fix.

All I can see for a past year or two in openFrameworks is, like [speculating], 60% of maintainer's time devoted for developing code and like 40% for maintaining makefiles / PG and solving bugs related to them. Half of the forum is filled not with questions like how to move objects in space or things like that related to oF code but with error reports of undefined symbols and compile time errors.

this is just not true, most of the time maintaining project files is spent on adding new libraries or new platforms like the current migration to 64bits, that's different on every platform no matter which system you use and wouldn't have been much less using cmake

Makefiles are slow. Like, super slow

what's the difference with cmake? seconds? milliseconds? they are also safer than cmake apparently, i've never had anything like: https://github.com/openframeworks/openFrameworks/issues/4177 with makefiles. also you would still to install ninja which is even one more dependency, cmake by itself would only add another level of indirection making it slower.

in any case, as i said before if someone can get cmake to generate project files using the current config files for addons and projects (ie: addon_config.mk, addons.config and config.make) and for all the platforms we use i think it would be worth because it would allow to use qt creator (and other others but i think that's the only IDE that makes sense right now apart from visual studio and xcode)

[PetrosKataras]

Personally I do not use any IDE and I have to say that I am really grateful for the current OF Makefile configuration since it works really fine for both Linux and OS X.

On the other hand as @procedural already mentioned I think with CMake we can have a more streamlined/unified approach for the building process under the various platforms. In addition since CMake is becoming a standard it would allow OF to be more easily integrated in external projects as a third party library ( like in the case of stardust for example ).

The good thing is that there is already a CMake basis to work with ( ofnode / stardust ) so we have something to start with. As I mentioned I have already started thinking / trying an integration of the current OF config by parsing the specific files in a similar style as it currently happens. The first idea would be to directly parse the files with CMake which it is a bit cumbersome because of the syntax and variable scope of CMake and the second idea I had was to call external scripts through CMake that would parse the files and echo back all the necessary variables.

[ghost]

as i said before if someone can get cmake to generate project files using the current config files for addons and projects (ie: addon_config.mk, addons.config and config.make) and for all the platforms we use i think it would be worth

Yea, that's just backwards, I can understand why it's needed for backwards compatibility, but it's not a topic I'm particularly interested in. I see @PetrosKataras has more to say on that

[ghost]

By the way,

apparently, there are problems with compiling some libraries, which can be solved with cmake

i don't think that's the case, constantine has been putting most time in this and he's been using cmake, i'd say that the same amount of time fixing our current system would have fixed the issue there too not that cmake makes it easier to fix.

Is true, giving more time I'm sure Apothecary can build libraries MinGW needs, I use them already not because of CMake, but because of MSYS2 package manager for Windows that provides most of MinGW and MinGW-w64 libraries for me. I build with CMake only those libraries which are rare to see in package managers, like Poco, or latest versions of which are not in package managers of long-term supported systems like Ubuntu 14.04.

[arturoc]

yes i saw that and it's interesting we could do with mingw something like what we have in linux a script that downloads all the necessary dependencies.

[ghost]

Also I just want to mention that not only Ninja is way faster build system than Make, even despite some quirks sometimes (#4177) (not always! that was the only case I can remember which forced me to recompile openFrameworks, and I'm still not sure whether Ninja caused this or Clang that uses GNU's standard library on Linux) it worth the speed that I've got for my projects, CMake also allows to do stuff like automatically precompile header files and use them without changing anything in projects, with Cotire module.

Here I wrote a post almost a year ago and it's still true today, I'm able to make a change in ofApp and recompile an app in less than a second. Of course it is possible to use this technique outside of CMake, but that will be just another layer of headache for people who will maintain that, compared to including a module and be ready to go with any IDE out there.

[ghost]

Who knows how many projects openFrameworks maintains will throw error because of spaces in paths: #4195. I solved it a year ago once in CMake files and never had it in any projects I generated since then. Add that to an ocean of pros for having CMake as a build system :)

[avilleret]

OF Makefile are very very very slow. Even more when native compiling on Raspberry Pi for example. It's anoying to wait 30 sec because I forgot a semicolon. So I take anything that can make build faster. I'm gonna test @procedural CMake system.

[ghost]

@avilleret note that I'm not compiling OF only, but some other libraries as well which are missed in package managers of popular OS's. Plus stuff like RPi and iOS / Android are not supported (due to lack of interest on my side), so it's not a silver bullet, but definitely a faster way to work with openFrameworks.

[like-a-bause]

Disclaimer: I am biased towards CMake and helped on getting ofnode running on OS X. To give this discussion a constructive spin again: The strongest plus for a transition to CMake would be simplicity. The guys working on CMake do nothing but making things compile on different platforms and IDE's. So the next time a new Visual Studio Version Pops up why let the OF community do all the work instead of letting the experts on that topic handle it? Second argument in that direction is lines of code. The more you got, the less maintainable it gets. I ran some numbers: openFrameworks (only including platforms i think ofnode supports as well)

find makefileCommon/ Makefile linux/config.linux.default.mk linux64/config.linux64.default.mk osx/config.osx.default.mk win_cb/config.win_cb.default.mk | xargs wc -l
wc: makefileCommon/: read: Is a directory
     347 makefileCommon//compile.core.mk
     459 makefileCommon//compile.project.mk
     196 makefileCommon//config.addons.mk
     423 makefileCommon//config.linux.common.mk
     512 makefileCommon//config.project.mk
     343 makefileCommon//config.shared.mk
       8 makefileCommon//Makefile.examples
      21 Makefile
      28 linux/config.linux.default.mk
      28 linux64/config.linux64.default.mk
     401 osx/config.osx.default.mk
     303 win_cb/config.win_cb.default.mk
    3069 total

of (anything i forgot?):

wc -l openFrameworks.cmake CMakeLists.txt
    1020 openFrameworks.cmake
     701 CMakeLists.txt
    1721 total

GNU people are the worst what could happen to open source community. Their licenses are weird, their code is unreadable, their tools are slow.

Without these GNU guys we would most definitely not be where we are now.

[ghost]

@aspeteRakete actually, openFrameworks.cmake and CMakeLists.txt duplicate half of the code between themselves internally... Ideally I need to find way to leave a single CMakeLists.txt file...

[ghost]

Without these GNU guys we would most definitely not be where we are now.

Not to diminish their accomplishments, but I don't find their tools usable or well written personally, at all. Good thing we have guys like Rich Felker on Linux or Martell Malone on Windows side whose hard work can improve the current situation.

[jvcleave]

@avilleret the OF makefiles are not the bottleneck - compiling on the RPi is. If you are doing any substantial amount of work on it a cross-compiler is necessary. You can actually set one up faster than OF takes to compile on the RPi. See here and this is the one I use. See the forums if you have any issues with these (and there are other solutions there)

[drlight-code]

I'm a strong believer in free software (mind the difference) and can't relate to any of the accuses of being slow and unusable. The target audience is not your casual developer, for sure, and sometimes you got to rtfm or even rtfsc quite intensively, but the fact that we are able to do the latter at all has to be credited to the free software movement to an extend you should not underestimate. If you want to improve the tools or write your own and do better, of course, some of those tools are aging and fresh ideas and concepts are always great. Regarding the philosophical question of licensing, people can be of different opinions, I'm very much in line with the FSF side in those regards.

That claim of yours that the GNU tools are slow (I guess you're relating to make?) I can not really comprehend, could you elaborate a bit on what aspects you relate to? I've written and maintained a plain Makefile based build system for a whole set of libs/applications part of a quite large software project, and for me (and others using that system) everything was always lightning fast and really smooth. You can't do things like including tons of headers from within headers or don't use proper include guards and expect otherwise, I don't need to tell you I suppose, but if you invest some time to get things right, the GNU tools worked really really well, at least in my experience. So I'm actually really interested in what you relate to specifically. Heavily templated code is a bit of a problem in terms of compilation times, but that's rather a conceptual thing in the language, and I don't know of any build systems which would be able to avoid that. I consider the overhead of GNU make rather minimal.

[avilleret]

I don't want to hurt anybody here. I just noticed that openFrameworks build system is slow. And when I talk about the build system I'm not saying that all build system based on Make are slow, but OF's one is. It could be improved, but I don't know how. For now, I'm using ofnode CMake system and it is much faster. I didn't do any benchmark, but I would one day if have time.

@jvcleave thanks for the hint, I'm already using crosscompiling for other project, I'll do for OF when needed. But in any case, a slow build system will be slow even when crosscompiling.

And again I never wanted to enter in a fight between GNU people and other or whatever. I would just to add my 2 cent to this discussion.

[drlight-code]

@avilleret Heya, well I might look into the reasons for that at some point when I find time for it, however I'm really busy here at work, so might take quite a while. No worries about any fighting, I think we are all professionals and lively discussion is nothing which will hurt anyone I suppose, even if there are basic philosophical discrepancies. I was specifically referring to @procedural's rather general remarks about the GNU project and the tools they developed. So easy going, no worries :)

[drlight-code]

@aspeteRakete fullack regarding your CMake remarks. I've worked with severl build systems, and although (or even because) CMake is still being developed actively, and backed by Kitware to an extent, it works really well if you get the hang of it, especially for reducing the cross-platform effort of maintaining several specific ones.

[danoli3]

If someones willing to maintain the CMakeLists, sounds great! CMake has saved so many projects builds with it's internal genius for me a few times, when all hope was lost.

[drlight-code]

I might give it a try, commit into a branch for people to test it, since it will take some while for everything to be smoothed out on all platforms. For now I'm still busy getting the openFrameworksConfig.cmake properly set up, s.t. at least external CMake projects can simply build against oF. However, work to do, so no hurries here..

[jvcleave]

@avilleret A cross-complier is about 15x times faster than native compiling. Cross-compiling current OF core takes <6 minutes compared to ~90 minutes natively on an RPi1.

Your statements that "OF Makefile are very very very slow" and "a slow build system will be slow even when crosscompiling" is attributing the slowness to the wrong part of the process.

[HalfdanJ]

New Android Studio 2.2 now supports CMake natively: http://android-developers.blogspot.com/2016/05/android-studio-22-preview-new-ui.html

[razvanilin]

Hey guys, what are your thoughts about moving towards cmake in the end? I was trying to add react-native to an oF app on Android and encountered some issues when I tried to compile RN with oF most likely because oF is still using gradle-experimental for building stuff and RN needs the normal gradle tools.

I opened an issue on the android-ndk asking for help and gradle-experimental appeared in the discussion a few times. Maybe it will be worth looking into replacing this with cmake, especially now that Android Studio has better support for it. Also, gradle-experimental is not supported anymore so not that future proof to continue with it.

[xarthurx]

Echo with this post, I'd also wonder if there's a plan to make OF as a library that can be easily added with a cmakeList to other projects.

OF is definitely a very useful platform in creative coding and interactive art, but still not enough when doing other things such as researches in robotic control and various HCI directions.

Adding support of CMake as a cross-platform library will definitely increase the impact and popularity in the academic field and attract more contributors from various fields.

[hiroMTB]

Cmake is not easy for programming beginner in my experience but defenitely useful in term of getting rid of effort to support multiple platform and put more resource into oF features.

So I think we should keep PG and also start supporting Cmake as an addon. (For example I started vscode oF support last year as an example project and now in master repo)

Here is ofxCmake. Half years ago I spent 3 hours to use ofxCmake, could not make it work that time. But it was almost okay I remember.

https://github.com/BildPeter/ofxCMake

[cerupcat]

I've also been trying to add some newer libraries to Android (ie. Realm Database, Oboe, etc) and they seem incompatible with using the experimental plugin (reference: https://github.com/realm/realm-java/issues/2539). Has anyone looked to moving off of the experimental plugin since that seems to be the common suggestion? I'm not fluent enough in the current build system to attempt it myself; it seems like it would need to be a larger effort.

My alternative is re-writing my app to not use OF.

@razvanilin did you find a solution for react native?