State of QBS (2.0)

[artificiel]

After being abandoned by the QT Group, QBS got pickup as a community-driven project which released a v2.0 in spring 2023 (and 2.1 in august). importantly, the scripting backend was ported from QtScript to Javascript, a move that is described as: (emphasis mine)

From a user's point of view, there should not be any noticeable changes; in particular, the performance appears to be about the same. However, QtScript was rather forgiving about accepting undefined values in contexts where it didn't have to, so the new implementation might uncover some glitches in your projects. https://www.qt.io/blog/qbs-2.0-released.

so I've recently unknowingly updated to that version and encountered undefined problems, and after spending a bit of time thinking I made something wrong, then a bit of time thinking my install was broken (I juggle many OF trees), it's only after reading the news above that it came together as it seems the OF module relies on undefined values by design, which means it's now broken. it does not seem so easy to fix/resolve (and as far as an hour wrestling with it goes) as "undefined" in javascript is actually a form of value and not the same as "null"...

I've always enjoyed QBS (as "enjoyable" as build systems can get) and did not mind doing some caretaking around it, but now with infrastructure change might warrant a deeper rewrite of the OF module, and since the official QT is now using CMake, and there are efforts to standardize CMake with OF in general, it goes without saying QBS should be left behind.

it is however important to support at least 2 IDE's per major platforms, so for linux it would make sense to maintain the QtCreator support, but with the CMake tools. if someone would take the challenge of integrating the PG and/or QtCreator wizard with the CMake OF, it would be super! but maybe that's a bit ambitious for 12.1?

In the meantime perhaps someone familiar with both old and new QBS could "port" the OF stuff but it's not clear if it's worth the effort if in a similar time frame QtCreator/CMake can be up and running?

there probably are not many apps relying on esoteric/advanced QBS features so the workaround is simply to switch to Makefiles, but it's important to give a heads up so QtCreator users can forecast trouble if they install QtCreator > march 2023 (i could not find the exact version number where QBS became 2.0).

lingering issues about QtCreator/QBS: #7248 #7021 #6733 #6420 #6237 #5897 #5349 #5173 #5167 #4987 #4763 #3136

[dimitre]

It is now easier to update projects in projectGenerator, The question is: Do we want to support it? If yes I can help with needed changes. I personally never used before, and now we have VSCode template as an alternative

[artificiel]

we probably want to distinguish "supporting the QtCreator IDE" and "supporting QBS".

the Qt Build System 2.0 is CMake-based and relies on javascript which is probably simpler to handle than the original custom language, but it would need a CMake-knowledgeable person to advise how to organize things in the template. (For sure, the QT Wizard is probably unneeded (NB: I've never used it) as the PG can do whatever path assembling is needed).

independently of the QBS, the QtCreator C++ IDE is good and if it's possible to access debugger integration, analysis, etc. it would be good to have access to when you have familiarity with it. Also incidentally QtCreator is cross-platform which means an option on all platforms. So if we can manage to have the IDE function properly it would be great.

so I guess it means: can PG generate a CMake OF project and what distinguishes a QBS 2.0 CMake project from a "plain" CMake project? can QtCreator manage "plain" Cmake projects? it would be the simplest.

[jcelerier]

so I guess it means: can PG generate a CMake OF project and what distinguishes a QBS 2.0 CMake project from a "plain" CMake project? can QtCreator manage "plain" Cmake projects? it would be the simplest.

QtCreator works just fine with plain Cmake projects, it provides pretty deep integration

[artificiel]

OK, good to know!

right now the CMake effort seems stalled by ideological friction around the handling of dependencies, which OF handles in it's own way (with Apothecary, to provide ready-to-go libs on all supported platforms binary dists), which makes sense historically but now there's options).

perhaps a way to circumvent that deadlock would be to have 2 layers of CMake: one simply as an app project file (which multiple people managed to do to solve their own problem, so it's a known-to-be-solvable problem), different from "handling the whole OF CI in CMake".

that means adapting the addons-parsing functionality of PG into producing a CMakeLists.txt, and depending on what is provided by download_libs and the contents of addons without trying to resolve/download dependencies in a wider/more modern manner.

I'm not involved in PG nor very with familiar with CMake so while I don't mind keeping abreast of developments I don't know what doing that entails (maybe the work of @HerrNamenlos123 already covers that aspect?). But certainly getting a basic CMakeLists generator that imports/builds projects against $OF_ROOT would be a great first step for users, until the more complex issue of dependency management is solved. The template could be titled CMake (local) or something like that, to distinguish from an eventual "total" CMake integration.

[artificiel]

also (just noting this here) getting CMake support would also facilitate VSCodium use because it provides better access than Makefile to a compilation database, which is required by the non-MS C++ extensions (or, at least, was required last year).

[HerrNamenlos123]

Firstly, you are right, CMake makes it very easy to use a Compilation database and yes, it is necessary for any IDE that does not habe its own build system. Specifically, even editors like NeoVim need it for LSP support. Unfortunately, Visual Studio does NOT produce compile_commands.json, all others do. If you want to work in Neovim with MSVC toolchain and still have LSP, you can use Clang-CL which is a drop-in llvm-based replacement for MSVC, which does provide compile_commands.json

[HerrNamenlos123]

And to CMake: my work more or less covers everything you mentioned, but it does not yet cover all options needed and it is significantly more complicated because of the old apothecary. new packages will simplify this.

The CMake in OF itself is more or less done, but it does not make sense to distribute it yet while the dependencies are still such a mess.

For packages: I am not sure what platforms and compilers are needed. The first packages compile fine using the mainstream compilers, even armv7 GCC cross compilation, but platforms like Windows Arm64EC, MinGW in general and clang aarch64 cross-compilation cause a lot of trouble. In that regard we should clearly define what compilers and architectures are needed, in order to finally get to a first working prototype and not end up adding every possible architecture.

[artificiel]

Understood about apothecary/dependencies (and that specific topic should be discussed elsewhere!)

But the idea proposed here is to make a "light" CMake approach that relies on the currently distributed libs structure. In short, have PG generate a minimal CMakeLists.txt based on whatever local info the PG gathers from OF_ROOT and the specified addons, that takes for granted the old-school download_libs approach — i.e. the generated CMakeLists might not be portable, just like the current Makefile or other IDE projects which absolutely point to $OF_ROOT and need to be re-generated if the context changes.

It is a stop-gap maneuver, to enable CMake for the end-user.

[HerrNamenlos123]

I see, sorry i thought this was the apothecary discussion, not QBS. There is a way we can temporarily fix the current libs approach so we can use CMake in the meantime, which i'm going to discuss in the Apothecary discussion.

[artificiel]

yes the discussion do overlap haha but to be clear (my grasp of CMake is superficial) I understand CMake likes to "find things" (so, presumably "discover/learn" what's provided in libs).

presumably for CMake to be progressively adopted it will need at least for a few years to support the current addon_config.mk format to import the information about an addon — of course the ideal world would be to addons to be CMake-friendly (somewhat like @peterkolski did here https://github.com/peterkolski/ofxCMake/tree/master/addOns, but we can't expect addons to suddenly sport a .cmake). So there needs to be a mechanism for a user to produce a CMakeLists that takes into account the "foreign" addons.make to get the list of addons, and then process the addon_config.mk to integrate the equivalent of those .cmake. maybe this can be performed natively in a smart CMakeLists, but currently that job is being handled by PG for all the supported platforms, relying on manually-user-provided OF_ROOT and producing relative or absolute paths. And note that ofxCMake uses main.cmake and config${PLATFORM}.cmake as core scaffolding; their equivalence could be provided in libs/openFrameworksCompiled/project/cmake, where PG can find them.

so I guess the question is: can PG generate a hard(er)-coded CMakeLists that will build a specific app, taking into account the same local context it has to assemble the other platform/IDEs files?

of course that would be a very inflexible CMakeLists, and that is probably counter-logic to the strengths of CMake, and it's not so satisfying to produce half-assed solutions... but from the perspective of the current user, a PG run is a known expectation, and PG being there with the logic of parsing a directory to generate different projects/IDEs, it might be a faster path to put a CMakeLists in end-users's hand for the purpose of compiling their app as an side-by-side alternative to Makefile, within the current (and familiar) binary distribution organization.

[HerrNamenlos123]

@artificiel I am not 100% sure what you mean regarding the PG generating hard-coded CMakeLists. But I assume you are talking about the procedure of CMake using find_package to locate dependencies. FYI the most CMake-friendly approach is to find all dependencies using find_package and providing the right module files so this approach works. All system packages do it this way. However, either as a commandline parameter or in CMakeLists.txt itself, you can set CMAKE_MODULE_PATH, which is the path it will search for dependencies. If you set it to just a single absolute filepath it will search ONLY this subfolder. Thus, using PG you can simply point CMake to a single folder and instead of finding system libs, it will find ONLY the libs downloaded by PG, or cause an error.

[artificiel]

aah sorry about the messy explanation, I'll rephrase: when starting work on an existing OF project (either something downloaded, or one's older project on a different platform, or same platform but different IDE etc) the operation is to open the project directory in Project Generator and choose which project type one wants to get generate. PG then scans src for source, integrates $OF_ROOT, and adjusts the project model with the contents of addon_config.mk for each addon listed in addons.make (includes, flags, data, etc). The generated project file is more or less static (and absolute) depending on the platform's (or IDE) features.

So a more focused question: considering the goal of having CMake as a PG target (presumably generating a CMakeLists.txt), what would it look like in terms of addon processing?

[HerrNamenlos123]

I cannot give a precise answer on that as I am too unfamiliar with the addon_config.mk fileformat. The addon itself can have the normal layout without any issues, and even if it's not exactly CMake's best practice, we can just add all files as source files in the src folder, etc. So addons without any dependencies should be trivial. I am not sure what to do with addons that have extra dependencies, however. As said I have no idea about the old Makefile approach and don't quite understand it: How would it even work in Visual Studio? Can VS integrate makefiles? I thought Makefiles are Linux exclusively??? I know that something like nmake exists, but I thought that's completely different. How can the current Makefile approach work for all supported compilers and build systems?

At first I thought the Addon.mk can be parsed and interpreted by CMake but I think this is a very, very bad idea. In the long run, every addon will need to provide a CMakeLists anyways, if it needs additional dependencies besides the normal src files. I mean the information about the dependencies needs to be available somewhere and AFAIK currently it's only in .mk, right? If it's easier for people, a small config file might be an option, like a simple text file or .toml file, where you list additional libraries, which is then parsed by CMake. It would mean people don't have to write scary CMake files. Besides that, you would have to manually add a library link for the libraries your addon needs.

One last idea: AFAIK Makefile is more or less just a scripting language.... What if it was possible to create a small script, that would run the Makefile and generate a small CMakeLists, which acts as a starting point? Then plugins could be converted much easier (although not reliably)

[artificiel]

ok I think we're putting the thumb on the topic!

addon_config.mk is custom to OF, and includes config info such as compiler flags, includes, data to be copied, etc. and is processed differently on different platforms — it's kind of a package description in a declarative format. addons.make is the list of addons that should be integrated in a project. These things have 'make'-like extensions, because the original setup was with Make, but it's parsed as text by Project Generator. perhaps a more modern format for addons could be adopted, that's open to discussion, but in any case the current format will need to be supported. some examples using different features:

https://github.com/danomatika/ofxMidi/blob/master/addon_config.mk https://github.com/leadedge/ofxNDI/blob/master/addon_config.mk https://github.com/zkmkarlsruhe/ofxTensorFlow2/blob/main/addon_config.mk

What if it was possible to create a small script, that would run the Makefile and generate a small CMakeLists, which acts as a starting point?

so this small script you're thinking about is exactly Project Generator! you give it a file path (where the source lives + addons.make), $OF_ROOT (where the specific version of OF you want to use lives (you can switch between OF version by pointing to different directories)) and it does, for any supported platform, the job of assembling the dependencies into a Project File. So it generates a native Xcode (iOS or Mac), or Visual Studio, or Makefile (or QBS — we remember the topic we're in haha) etc. Now we want it to generate a CMakeList.txt.

At first I thought the Addon.mk can be parsed and interpreted by CMake but I think this is a very, very bad idea. In the long run, every addon will need to provide a CMakeLists anyways

we cannot assume all addons will have a CMakeLists on day 0 of CMake, so generating an add-on's CMakeList based on the existing addon_config.mk is a requirement. Project Generator already knows how to do the work and it's a simple matter of assembling/managing text templates (and people will be able to help; @dimitre has been busy in PG lately) but we need guidance on what to generate:

• what is the minimal CMakeLists that will compile an app taking for granted only the current structure of $OF_ROOT (provides Debug vs Release)
• how would a CMakeList for an addon look, if one needs to:
  • add src (.h, .cpp from $OF_ROOT/addons/addon/src)
  • add flags to linker, compiler, include or other (NB sometimes libraries are binary-provided in addons)
  • copy data from the $OF_ROOT/addons/addon/data dir into the app's bin/data dir (e.g. fonts, shaders, or other assets)
  • sometimes copy $OF_ROOT/addons/addon/libs/.dylib or .so (I personally try to use .a to diminish floating dependencies but sometimes not possible)
  • can app+addons be concatenated in a single CMakeLists?

again, that what I meant about a stop-gap — not ideal, but will be required as there are hundreds of public addons out there (who know how many private ones; I have a few dozen myself, some with custom flags or libraries) — and even more projects depending on this configuration approach.

also to be clear, Makefile is not linux-only; for instance on macOS I can have PG provide an Xcode project, or Makefiles — both will compile the project. It's also the core of the current VSCode cross-platform support.

[Daandelange]

Hey, Sorry to jump in only now. I've been using Qt-creator on osx and linux since the beginning and I'm still very happy with the current state. Meanwhile, QtCreator support could use some updates, as initiated in #5897. It's still my main IDE for OpenFrameworks.

(correct me if I'm wrong)
Currently Qt-creator is locked at version 4.x (I have 4.15), later versions break OpenFrameworks' .qbs files.
So is this discussion about migrating it to the recent QtCreator IDE which embeds QBS 2.0 ?

I can contribute some extra knowledge : The current QtCreator build system has some QBS1+JS scripts to extract the build information from the addon_config.mk files; the code is quite obvious and corresponds to «generating a CMakeList from addon.mk» which you mention above, but for QBS 1. The parser helper is already in JS so that might be useful for porting it to QBS2.
I think porting the current version will not be too hard; the qbs syntax changes but (I guess) the properties to set remain similar. I'm willing to try if that's the direction to take.

On the other side, it can be a headache to keep 2 different build-system-parsing-behaviours in sync, and cause issues if differences exist (but the same is true for ProjectGeneratored CMake files).
From my experience coding with QBS and ofxAddons, I ran into a lot of compilation issues, but the advantage is that it's flexible enough to create "dirty" fixes within the .qbs file to overcome problematic addons while still remaining compatible with others : you can embed the myOfApp.qbs and others are ready to compile. (see this sample in ofxSyphon)

Anyways, what I like the most about qt-creator is the dynamic reloading of the quite modular/customisable .qbs file, without needing to use the PG, like the makefile system, but more modular. IMHO it'd be great to have a "dynamic addons list" if possible in CMake, rather than having the PG "hardcoding" them once parsed.
For example: Update add a file to an ofxAddon and no need to regenerate all your project files. Same if you add an addon to your project.

[artificiel]

OK! so i re-read the whole thread and to synthesize and answer your direct question:

So is this discussion about migrating it to the recent QtCreator IDE which embeds QBS 2.0?

maybe!

1) proper QBS 2.0 support would be great (and i agree that managing addons and options in .qbs is a very fluid build system)

but:

2) (here my CMake shortcomings do not enable me to be more assertive in terms of what is the ideal plan) because QtCreator apparently has good CMake support as project format, could a generic OF CMake project file cover both the usage in QtCreator, as well as plain CMake usage? this would be "two for one" and reduce things to maintain (and also maybe even VSCode support would beneficiate from that, as well as potential unknown IDEs) BUT this presumes that QtCreator is able to react as dyamically to a change in a CMake project is well as it does to .qbs. (e.g. adding or upgrading an addon by editing the file, hitting ^-s and seeing the source tree get re-parsed)

so the last part of my last reply is pointing back to PG (which already knows how to interpret addon_config) in order to generate a kind of "intermediary" CMake project that could rely on the current deterministically apothecary-delivered core binary libs (as opposed to an holy grail CMake project that starts on a blank machine and resolves and retrieves all OF and addons dependencies as conan packages or something).

[Daandelange]

Ok, thanks for the sumup.

Yes, ideally CMake sounds awesome on all points, also for compatibility with almost any foreign IDE. (me too, I presume IDEs that support CMake do auto refresh on change) But what about CMake support then ? There have been many drafts around, but nothing ever got merged. The latest one being #7524 (?). I'm not familiar enough with Cmake too. I'll look into updating the QBS files to evaluate the workload, I feel it might be worth updating QtCreator soon or we might lose this IDE until some CMake support is provided.

Something else I thought of, regarding the need of the PG to update project files (CmakeList), a watcher could also be an option, like npm run dev (Electron//PG) : projectgenerator watch path/to/of.

Edit: QtCreator 11.x is the latest supported version on osx 10.15, meaning the IDE will already be dead on some of the supported OF platforms.

[artificiel]

I haven't closely followed the CMake status, but as I understand: there are is work to be done on the apothecary side for things to fall in place (it's being converted to CMake, e.g. https://github.com/openframeworks/apothecary/tree/bleeding/apothecary/formulas/cairo)

on the CMake "project file" side there are some top-level design decisions to be made and @HerrNamenlos123 above did good work on core CMake support, but it seems now the point of friction is how to handle addons. the point of my comment just before yours was to broadly summarize the requirements of "whatever" in regards to interpreting addon_config (because existing addons will not suddenly be equipped with a CMakeLists config).

I guess what's missing is someone equally at ease with PG/addon principles and CMake and I'm certain the technical work per se is not difficult; it's more about find the right compromises. or maybe that someone exists but the priority is on what I referred to as the "hole grail" where CMake is not only a build system but a dependency discovery/procurement system.

[roymacdonald]

I have heard some really good things about Cmake and I think it would be great to adopt it because it would allows us to not worry about all the different IDEs. (I guess), as long as you have a well formed and valid cmake file you can generate the projects for any of its supported ides. it sounds like a win-win situation. I have not diged into it but it would certainly be a good way to go.