Proposal: Split backends into separate build2 packages

[Rookfighter]

Hi there,

thanks for providing the imgui build2 package. I was just wondering if it would make sense to split the package into multiple packages, say a core imgui package and then one for each backend, e.g. imgui-glfw, imgui-opengl, imgui-sdl, imgui-vulkan, etc. This would basically eliminate the need to pull in all dependencies at once like glfw or sdl (when it's available as package).

If you like the idea, I can happily create a PR with a initial proposal.

[Swat-SomeBug]

Hi @Rookfighter I think splitting the package up currently is good solution. Ideally I would like to have a single package having optional dependencies that can be fully configured with variables. As I understand, such a feature is in works. But currently we don’t have another solution. @boris-kolpackov are optional dependencies on the roadmap for 0.15.0?

May I suggest we split the package only minimally where imgui depends on other packages? As an example, several rendering backends are expected to be available on the system while platform renderers that require external dependencies are glfw, sdl and glut. So I think splitting the package up into 4 packages (3 requiring external deps + 1 generic) would be sufficient.

[boris-kolpackov]

@Swat-SomeBug

are optional dependencies on the roadmap for 0.15.0?

Yes, conditional dependencies (i.e., the ability to say "I only need this dependency if feature X is enables, building on platform Y, etc") are already implemented (and available in the staged version of the toolchain). And we are working on the dependency configuration (i.e., the ability to say "I need libfoo with features X and Y") as we speak.

In this context, I would actually be interested to understand what's going on here so we can make sure this use-case is covered. Am I correct that any application that wishes to use imgui must also link a backend (I presume the backend performs the actual drawing)? Or can it just ignore the backend so that an appropriate one is linked automatically somehow? Can there be a case where someone wishes to link multiple backends?

May I suggest we split the package only minimally where imgui depends on other packages?

While I think not making things too granular is a good idea, on the other hand, compiling and linking code that is not needed could also be a drawback. One compromise could be a single package that provided several libraries (core plus the dependency-free backends). The application can then import (and thus build/link) only what it needs.

[Klaim]

Am I correct that any application that wishes to use imgui must also link a backend (I presume the backend performs the actual drawing)?

My understanding (maybe something have changed but I doubt it) is that DearIMGUI will provide raw data that have to be used by whatever rendering system you want to use, even a custom one, to perform the drawing and push input events into the library. But because some rendering and input APIs are very common (OpenGL/Vulkan/DirectX/Metal with/without SDL2/SFML/... etc.) the repository also contain backends to easily integrate the library directly with the API you already use (because the whole tool is designed to add tooling inside a game that's already working - although it is general enough to be used as just another GUI library).

Can there be a case where someone wishes to link multiple backends?

This happens when games use different backends for different target platforms (compile-time choice) or when they allow users to chose the backend (runtime). This is not super common these days, but it's still happening (mostly with big companies who can afford to do that or need the extra performance for the first case, and some games switching from one backend to a new one for the second case, often vulkan - it was also common in the past to let users select DirectX or OGL on Windows because both had strong tradeoff depending on user's setup, but it's rare these days).

[boris-kolpackov]

DearIMGUI will provide raw data that have to be used by whatever rendering system you want to use, even a custom one, to perform the drawing and push input events into the library. But because some rendering and input APIs are very common ([...]) the repository also contain backends to easily integrate the library directly with the API you already use ([...]).

Ok, let me know if I got anything wrong: The application is expected to link to the core library plus 0 or more backends and, I presume, must "connect" the core with the desired backend at runtime (or, alternatively, do the rendering itself without a backend).

[Klaim]

I believe this is correct, maybe @Swat-SomeBug can confirm my understanding?

[Swat-SomeBug]

@boris-kolpackov and @Klaim Yes this is correct. imgui itself distinguishes 2 types of backends. The platform backend is used for window related activities including input from mouse/keyboard, while the render backend is used to actually the graphics API used. There is also support for plugging in higher level frameworks that perform both tasks.

The render backends are low level APIs such as DirectX or OpenGL and this package assumes these are preinstalled and are system dependencies.

Platform backends typically are also provided by the platform on which imgui and its applications will be built. Examples are native windowing systems on Windows or MacOs. However, there are additional support for 3rd party windowing + input/output systems if package consumers wish to use those libraries instead of the native system libs. Example, glfw which is cross platform or sdl that is even more than just a window lib.

Package consumers need to specify which Platform backend and Render Backend they would like to choose (not all combinations are possible), and the package should link against this choice.

In this context, I would actually be interested to understand what's going on here so we can make sure this use-case is covered.

The use case for optional dependency comes from the use of 3rd party libraries. The idea would be to allow download + build of these 3rd party libs depending on user provided configuration variables. imgui in particular defines 2 config variables config.imgui.platformbackend and config.imgui.renderbackend and has some logic to select the right libs needed. If optional dependencies are possible, the wish would be to define them in the manifest along with a boolean value evaluated in root.build. The dependency should then be built just as another build2 package.

Particular example for imgui would look like something like the below code

manifest
---------

depends: ($config.imgui.platformbackend == "glfw") ?glfw

This makes glfw an optional dependency if and only if $config.imgui.platformbackend == "glfw" If the expression evaluates to false, then glfw is not downloaded and built.

When this is already supported by build2 then there would be no need to split imgui.

[Klaim]

When this is already supported by build2 then there would be no need to split imgui.

You can already try it in the current stage version (v0.15.0-). @boris-kolpackov gave me this example in the past (I don't think it's documented yet?):

depends: dear-imgue-directx ? ($cxx.target.class == 'windows') config.myproject.backend=directx

I guess adapted to your example, it would look like that?

depends: ?glfw ? ($config.imgui.platformbackend == "glfw")

---

Note that if we want this package to support the case where the end-user application needs to choose the backend at runtime, then

• instead of checking for equality for config.imgui.platformbackend we might want to check if it "contains" the string instead, allowing it to have multiple values;
• we would need to provide different targets for different backends as @boris-kolpackov suggested and let the user link to multiple of them as needed - but only provide the ones which are related to the backends listed in the config;

OR we don't allow this case and only provide configuration/build-time choice of backends. Or it could be a first step and wait for people to requestion the runtime choice to do more.

[Swat-SomeBug]

Thanks @Klaim!!

@Rookfighter I think we should try using optional dependencies from the staged version of build2, to help avoid needing to download + build unnecessary packages. For the moment, I think keeping things simple and allowing build time choice is good enough for 90% of the people. For more advanced options, consumers can definitely set the both config.imgui.platformbackend and config.imgui.renderbackend to empty strings and add any custom linking strategy they wish, including using other backends! This should already be possible (modulo any bugs) and is documented in the README.

[Rookfighter]

@Swat-SomeBug Sounds good to me, I would prefer to use the most idiomatic build2 way of things, so optional dependencies sounds like the right choice here. I can work on a proposal in a few days and file a PR. If you feel like doing this ASAP, feel free to create the proposal first, since I am currently busy and will have only time next week to work on this.

[boris-kolpackov]

Sorry for the delay in commenting on this.

When this is already supported by build2 then there would be no need to split imgui.

I think this is not as clear cut. Yes, you should be able to handle this with a single package, config.imgui.platformbackend and config.imgui.renderbackend, conditional dependencies (on imgui side), and dependency configuration (on the user side). And that would be the approach if imgui had to have its backends built-in into the single monolith library.

But, if I understood this correctly, backends are separate libraries and the user has to "glue" them together in their application anyway. So instead of the single package approach we could split them into separate packages. This way we don't need config.imgui.* nor conditional dependencies and the user can just use straightforward depends: for the backends they need instead of messing with dependency configuration.

As an aside, while the conditional dependencies turned out pretty straightforward, dependency configuration is proving quite complex. We've been banging our heads against this for the past couple of months and while we think we've finally figured most this stuff out, we are still some way from being done. In particular, while the dependency configuration for a boolean feature will be pretty straightforward (you just set it to true), for something more complex like the list of backends, the user project will have to cooperatively negotiate the configuration with other packages that may also have an opinion on which backends should be included. To put it another way, the only package manager that I am aware of that supports dependency configuration is Rust's Cargo and they only support boolean features, which on the surface looks very restrictive. But now we understand why (but will still support non-boolean configuration).

So I think what this all means is that we have two possible ways to handle it:

1. As a single package with config.imgui.*, etc., and where the user will have to jump through quite a few error-prone hoops to negotiate the backends.

2. As multiple packages where the user specifies the backends with simple depends:.

Or, a variant of 2:

2a. As a core package that includes all the dependency-free backends as separate libraries (so they are not built if not used) plus additional packages for other backends (one package per backend).

Seeing that this is a fairly popular library, I personally feel that we should try to make consuming imgui as straightforward for the user as possible and got with option 2 or 2a.

Thoughts?

[Klaim]

Solution 2 and 2a appear more "flexible" to me as a user, and easier to maintain on the long-term. Though I'm not the main maintainer so I might not be seeing some complications.

[boris-kolpackov]

@Swat-SomeBug Any thoughts on this?

[Swat-SomeBug]

Hi sorry for not getting back to this earlier.

2a. As a core package that includes all the dependency-free backends as separate libraries (so they are not built if not used) plus additional packages for other backends (one package per backend).

I thought of this for sometime and as I understand it, the core is extremely tiny and allows one to integrate into any backend including a custom engine.

Currently supported backends (where consumers can just copy out the code and hack as they like) are only for quick reuse. This kind of setup then really feels like your 2a proposal and doesn’t really feel like Imgui has “optional dependencies” from a core perspective itself.

So I would say I’m in favor of the split in general and I like @boris-kolpackov’s 2a scheme.

Proposal:

• Core Package Iibimgui with target lib{imgui}
• Separate packages for each combination of platform and render backends where it makes sense. There are 6 generic platform backends and 10 render backends and a very specific platform + render backend combination. So the total package count has to be < 60. The number would be much lower considering not all combinations are possible.
• We always can start with a minimum set of packages and create more based on community interest.
• Naming convention for these additional backend packages libimgui-backend-<platform backend>-<render backend>. Example libimgui-backend-glfw-opengl3.

[Swat-SomeBug]

@Rookfighter if you already had a proposal or PR, please feel free to submit it. Would be great to hear your vision as well.

[Rookfighter]

@Swat-SomeBug I have not worked on a proposal yet as I wanted to wait where this discussion leads.

I agree with @boris-kolpackov: optional dependencies probably make the configuration of a package quite complex and users would need to look into the package build code to actually understand which parameters they have to set. So I would also prefer the 2a approach.

Now about the proposal of backend packages like libimgui-backend-glfw-opengl3, I think this might lead to a combinatorical explosion of packages. Just let me know if misunderstood things: but this approach would mean we would have to have a package for each render backend and platform backend combination, correct? As long as render and platform backends are compatible, of course.

Now, I am not too familiar with the backend handling of imgui, but do we have to explicitly build render and platform backend together? Or would it be possible to have a package libimgui-backend-glfw and a separate package libimgui-backend-opengl3? Users could then just depend on the two packages and link them.

Maybe this is just the point where we have to implement something to get a more concrete picture of which approach makes sense and which doesn't. I will have some spare time the next days, so I would just start working on something and file a PR.

I also worked on a similar build2 package during the last week based on TGUI (see https://github.com/Rookfighter/tgui). It is also a GUI framework with exchangeable backends, but there I just figured that the library and the corresponding backend have to be built altogether. The library is filled with backend-related #ifdef even in header files, so it was impossible to extract any kind of independent core library. Not sure if this is the same with imgui. Any experience with that? Btw, I have not yet found a solution that I find satisfactory.

[Klaim]

This is also how I understood the proposition, have 1 package per backend (system or render) and then let the user chose what they need, link what they want and how they want. The end-user always have to add some calls to these anyway.

Then probably each backend package need to set some optional dependency requirements? Like if I use the SDL package, I can probably also use the OGL package but the version of ogl supported by SDL better be constrained in that case - and maybe not SDL versions are supported etc.. If proper requirements are set, then specific cases that would not work can be detected.

I would like anyone to be able to also link backends if they want to. Letting users depend on whatever set of backend they want allows that.

Personally I would prefer a naming like libimgui-render-vulkan for rendering and libimgui-system-win32 for system backends.

(by the way, the current name is Dear IMGUI, I'm not sure if it's a good idea to keep names like libimgui, but that's a minor concern, almost everybody familiar with imgui libraries knows what we are talking about)

The library is filled with backend-related #ifdef even in header files, so it was impossible to extract any kind of independent core library. Not sure if this is the same with imgui.

No, as far as I know IMGUI's core code is very separate from any of the backend, it just relies on the graphic data and custom event values as I/O between the two parts. I might be wrong, maybe something changed in the recent years but my understanding is that the author is very strict on that point (for good reasons: it's used on closed hardware by people working on custom game engines, it cannot escape this constraint because that's what made it work).

[boris-kolpackov]

Thanks for the feedback everyone!

I think the remaining fuzzy point (at least to me) is whether render and platform backends are "fused" together or whether they can be independent and the user can pick and choose. What could potentially clarify this issue is whether there is a dependency between render and platform backends. For example, in libimgui-backend-glfw-opengl3 (where glfw is the platform and opengl3 is the render), who depends on whom, if at all? I am guessing glfw depends on opengl3. If that's the case, are there alternatives to opengl3 that glfw could depend on instead?

by the way, the current name is Dear IMGUI, I'm not sure if it's a good idea to keep names like libimgui [...]

Debian calls it libimgui: https://packages.debian.org/bullseye/libimgui-dev So I think we are good here.

I also worked on a similar build2 package during the last week based on TGUI (see https://github.com/Rookfighter/tgui). It is also a GUI framework with exchangeable backends, but there I just figured that the library and the corresponding backend have to be built altogether.

I think this is the case (1) I described above where you have a monolithic library with several backend options. We can discuss it in more detail if you would like, but I think it's better to do this in a seperate place (maybe an issue in TGUI) not to have two parallel discussions here.

[Klaim]

For example, in libimgui-backend-glfw-opengl3 (where glfw is the platform and opengl3 is the render), who depends on whom, if at all? I am guessing glfw depends on opengl3. If that's the case, are there alternatives to opengl3 that glfw could depend on instead?

I believe it's always platforms potentially constraining renderer. This also matches the examples provided by imgui, you can see different platforms/frameworks/watever-system-abstraction with various renderers or none specific.

For example when using SDL2 you can decide to not use the SDL2 graphic API and just use OpenGL2/3 or Vulkan directly, in which case the only common data between these is the window "handle" (type dependent on the platform) which will be the same for both, probably created by SDL2 and then passed to whatever rendering api. So here no real constraint yet. However if you use SDL2 graphic API, you need to specify which rendering API you want to use too (even with emscripten) and then pass the proper data specific to that rendering API to SDL2 so that it can use it in it's implementation. At this point only a few rendering APIs are possible to use. So it looks to me that it's an optional dependency requirement of some sort, that would come from SDL2. You can also just use the renderer provided by SDL2 itself (see the sdl2_renderer example). It seems to me that these constraints would come from the SDL2 package itself, not the imgui-sdl2 package?

(note: as a real example, I once worked on a game where the rendering engine was Ogre3D, which let you chose statically or dynamically which rendering API you want to use - I was using OpenGl3 but I was using SDL2's windowing and input API and Imgui, so I was passing the window handle from SDL2 to both Ogre3D and IMGUI and the OpengGL3 and SDL2 backends for IMGUI - this is a setup actually pretty common in games not using generic engines)

Looking at the backends and the examples (which are very well documented by the way) we can see that there is no direct relationship between renderer backends and platform backends except a comment that tells us to use it in combination to one of the other kind.

Then in the end-user code we can se that it is where the mix is really happening. Taking your glfw example, you can already see 4 examples with different renderers using glfw. In every cases, headers from both backend have to be included by the end-user code in addition to the core of imgui to be able to work with them. Then the user code initialize each system and imgui, then it have to initialize the backends with the data from the related systems. Imgui works "frame by frame" and decides what it should display depending on the difference of internal state between these frames, so it have an api where you need to specify when a frame starts and when it finishes and as a user every other kinds of calls you do to imgui (for example to display a button in a window) is done between these begin/end of frames. The backends need this info too so the user also needs to call begin/end of frames specific to these backends.

This is quite well done in my opinion, the separation seems total and the mix needs to be done at the end-user level, always. (if there are exceptions I don't know them).

Also note the example_null project which uses imgui without any backends. While it's not really useful in practice, it kind of show how the separation is hard between backends and imgui itself. (comments says it's used for testing)

[boris-kolpackov]

This is quite well done in my opinion, the separation seems total and the mix needs to be done at the end-user level, always. (if there are exceptions I don't know them).

Thanks for the clarifications. I think you are right, I browsed a few backends and there are no dependencies between them whatsoever. For example, even the SDL (platform backend) and SDLRenderer (render backend) are not aware of each other.

If this is correct, then it sounds like our job is pretty straightforward: package each backend into a separate package. Each such package will depend on imgui core plus whatever backend-specific dependencies (glfw, sdl, etc). In fact, seeing that there are only 17 backends, I wonder if it makes sense to go with option 2 rather than 2a and just always put a backend into a separate package. This way the user doesn't need to figure out whether they need to add a dependency of if the backend is part of the core.

[Swat-SomeBug]

One point that still concerns me is how to package tests/examples in this case. Upstream has example_<platformbackend>_<renderbackend> for each combination. Should we also do this? So this repo would basically have a core imgui lib package + platform backend packages + render backend packages + multiple samples packages for each combination like upstream?

[Klaim]

One point that still concerns me is how to package tests/examples in this case.

Good question. One package with all the examples would mean that that package requires all the backends. I dont think it's worth doing that, making one small package each would work better.

Note that it's less a priority than the backends and tests, it could wait.

By the way, if my understanding is correct most example packages will not work on the CI anyway as there is no windowing or graphic apis available in there at runtime... They are only useful for end-users to try imgui and look at the code to use it.

Imgui can be tested with the tests and the example_null.

So this repo would basically have a core imgui lib package + platform backend packages + render backend packages + multiple samples packages for each combination like upstream?

Yes, the repository could be organized like that:

imgui/<here the `imgui` package>
backends/<one directory per backend package>
examples/<one directory per example package>
...<other files for a multi-package repo, see `bdep new -t empty`>

The packages.build file would then refer to the packages paths.

As examples are basically 1 buildfile and 1 cpp file (most of them at least) and just makes exes, I would recommend making the smallest kind of packages with just 1 directory, something like bdep new -t exe,no-subdir -l c++,cpp --package or something similar, then just replace the cpp.

[boris-kolpackov]

Hm, creating a package per example feels excessive.

Seeing that examples are a "leaf" package (i.e., no other package would depend on it), perhaps it makes sense to have a single package with all the example and a bool configuration variable to enable each of them (and use conditional dependencies based on these variables to pull in the backends). Then from the user perspective, it will be just:

bdep init ... config.libimgui_examples.sdl_opengl3=true

[Klaim]

Why not.

As long as it's possible to select which ones to try I guess that's ok.

[Klaim]

Work is happening in this PR: https://github.com/build2-packaging/imgui/pull/2

[Swat-SomeBug]

Fixed by merging https://github.com/build2-packaging/imgui/pull/2#issue-1257071005