Making Cinder more developer-friendly

[IntellectualKitty]

There are two main areas of difficulty when working with Cinder.

First, long compile and link times.

Second, the embedded libraries (1) replicate what may often already be on a developer's system, (2) they are difficult to update and maintain in Cinder itself, and (3) they substantially contribute to the long compile and link times.

For instance, on my system I already have the full distribution of boost, freetype, glm, AntTweakBar, Ogg-Vorbis, ZLib, and probably other libraries as well that I already use with other projects.

One of the annoying things about Cinder is that the embedded nature of the external libraries makes them difficult to update. And I want to be able to take advantage of the latest features and bug fixes. And it's particularly frustrating when I already have these updated libraries installed on my system but I cannot use them in Cinder.

A nicer approach would be to (1) allow the dependent libraries to remain external, (2) break Cinder up into modules, and (3) possibly convert Cinder's internal code into a header-only library.

As a standard engineering approach, decoupling generally makes for lighter, easier to handle, more adaptable code. Decoupling the dependent libraries from Cinder reduces Cinder's footprint, allows developers to benefit more easily from updates to the dependent libraries, and the reduced complexity makes it easier to expand Cinder and make it more adaptable and more powerful.

As another standard engineering approach, modular design simplifies and streamlines the code base, clarifies relationships between components, and makes components more user-friendly, user-accessible, and extensible.

To put this in more tangible terms, since I already have the aforementioned libraries on my system, it would be ideal to incorporate Cinder into my projects in the form of header-only modules. As an example, depending on the project I might need the application module, text and general graphics handling, image loading, and logging, but I don't need the extra weight of image processing, vector graphics, audio and video handling, GL interface, etc.

While incorporating all of Cinder's dependent libraries does make it easier for a novice to get up and running, it tends to be limiting for more experienced developers. Where Cinder shines, in addition to its easy to use programming interfaces, is the integration of so many powerful libraries. However, by separating them from Cinder and allowing them to remain external, the power of Cinder would be highlighted. Also, the reduced complexity and resultant flexibility would make it easier to make Cinder even more powerful, for example making it simpler and easier to incorporate different compression or image loading libraries.

[paulhoux]

Header-only implementations are a nightmare to maintain, due (among other things) to the enormously increased compilation times. A single change in a header might cause all code to be recompiled.

[IntellectualKitty]

This is from Wikipedia:

Header-only libraries do not need to be separately compiled, packaged and installed in order to be used. All that is required is to point the compiler at the location of the headers (the -I switch in gcc/g++), and then #include the header files into the application source. Another advantage is that the compiler's optimizer can do a much better job when all the library's source code is available.

The disadvantages include:

brittleness – most changes to the library will require recompilation of all compilation units using that library longer compilation times – the compilation unit must see the implementation of all components in the included files, rather than just their interfaces code-bloat (this may be disputed) – the necessary use of inline statements in non-class functions can lead to code bloat by over-inlining. Nonetheless, the header-only form is popular because it avoids the (often much more serious) problem of packaging.

For templates, including the definitions in header is the only way to compile, since the compiler needs to know the full definition of the templates in order to instantiate.

I am aware of the disadvantages, but the reason that I suggested the possibility of using the header-only library approach is that it tends to promote cleaner, more well-thought-out design and implementation.

I think that GLM is an excellent example of the advantages of using the header-only approach. It has a lot of functionality that is both well-designed and modular, making it easy for the developer to include (or not) as s/he sees fit.

[richardeakin]

Thanks @IntellectualKitty for the feedback. Regardless of pushing as much as possible to translation units to decrease compile times, I completely disagree that a header only framework can be written in a cleaner and more maintainable manner than how cinder is written, in fact I don't even think it would be possible. It would mean that your app and many other files in your project included windows.h, which causes a huge mess of problems. We use translation units to encapsulate the nasty details in the reality of application code, or video, or audio, or networking, whatever. GLM has a very direct purpose in that it is a generic math library, so it doesn't have to be concerned with encapsulation in the same way.

Concerning third-party libraries, we are moving to a more modular approach that allows you to exclude things like ci::audio or gstreamer from libcinder, particularly on linux distros where it is the norm to use package managers. I think we can do this for oggvorbis and libz, though by default, we'll always build these directly into cinder so they are available on all platforms. As the person who chose to include oggvorbis sources directly, and stand by this decision because it has greatly simplified my life as a maintainer. On any platform we port cinder to, we automatically have support for at least one soundfile type and there are no extra steps other than adding the oggvorbis .c files to that platform's build system and a couple include paths. We don't have to ship libraries for it for x64 + x86 + debug + release + shared + iOS whatever + every other type of configuration, and we don't need any extra install steps other than building cinder. This is especially valuable on Windows, where most of the core cinder developers spend the majority of their work days.

If you're hitting linker issues around any of our third party dependencies, then please open separate, specific issues, and PRs are welcome here too

We indeed care about having a decoupled set of tools and libraries, which I believe we've achieved. We also care about project setup times, since many of us use cinder on short time scale client work where time is very limited. In these cases it is great to have all the tools we rely on right there, within the master branch of cinder.

[richardeakin]

Concerning AntTweakBar / InterfaceGl, I'd like to see it move to the blocks folder but this will take a bit of work to update samples that use it too. We've also discussed shipping Dear-ImGui as an official cinderblock (lives in the cinder blocks folder), we even discussed moving samples to using this but I can't see us getting away from having InterfaceGl somewhere in the repo for legacy apps.

[PetrosKataras]

I think #2006 is also relevant for the discussion and indicates interest to address some of these topics. It also highlights Cinder's modularity to a certain degree, I believe, considering how straightforward it was to add this specific option ( albeit only CMake for now ).

[IntellectualKitty]

I think what @PetrosKataras is proposing is a nice compromise. In addition to building only those modules that are selected, perhaps option(s) could be added to use the pre-existing libraries on the developer's system as well.

[MikeGitb]

As a related observation: Last time I checked, one of the problems with "only building what you need" was that you still got include path conflicts if you are not carefull (I experimented a bit with making cinder compatible with vcpkg). Afaik the only reliable solution was to completely remove all the libraries and their headers from cinder's source directory structure.

[Hperigo]

This is a really good discussion, thanks for bringing it up @IntellectualKitty and here are some thoughts:

Having separate libraries would be nice indeed ( something like pex? ), I can image cases where just including ci::geom, ci::Color, ci::audio, ci::signals is nice without the whole cinder infrastructure. But to be honest without a package manager I have the feeling that it would be a pain to get things working in a somewhat straight forward way ( like @MikeGitb mentioned ), and it’s probably a lot of code rewriting too

BUT what I really wanted to talk is about Tinderbox. It is somewhat of a blackbox that does not get a lot of attention

Recently I wrote a python script that generates the same tinderbox folder structure but with a cmake project. From there I believe cmake can generate the correct xcode and visual studio projects if needed.

The advantage of having a new tool as a python script ( or whatever language ) is that wouldn’t require someone to know QT to contribute and beginners are somewhat comfortable using the command line to generate projects ( thanks javascript/npm ).

Some other features I can think right now:

1. Update project block list ( I think that is super easy with cmake, but a pain in xcode & visual studio)

2. Create a project using a already installed version of cinder ( usual tinderbox way )

3. Download a thin layer of cinder ( without samples, blocks or another file that is not required ) and create a folder with that cinder version and a another client/project app with the specified name ( see this for an example)

4. Create a block with XYZ name in the blocks folder with a basic sample, xml info, cmake stuff, etc…

Anyway, like I said, I wrote a basic script in a afternoon to play around with cmake and clion, I’m not sure of how much details/nuances would be in making a full replacement for tinderbox, but I think it’s worth the effort

[MikeGitb]

Personally, I don't see a reason to split up the different parts of cinder (like ci::adio etc) most of the times you need multiple parts anyway and I don't see what harm it does to provide everything together in one package.

However, in order to provide cinder with a package manager and generally prevent any conflicts with system libraries, it would be helpful if all 3rd party dependencies babe optional. E.g. just like boost they could be placed in their own git repository.

I have to admit though that I don't know how to tell visual studio to build only components, if they are there and use a package manager otherwise (it's easy in cmake though)

[IntellectualKitty]

@Hperigo I loved the link to Pex! I think that makes it really flexible, lightweight, and easy to use.

The machinery in ci::geom is very good, very powerful. But the organization – lumping everything into a single file – is the problem. If it were broken up into classes, e.g., Source, Rect, Sphere, Teapot, etc., it would make it far easier to work with, understand, and extend.

In contrast, the ci::ip library is extremely easy to use. Everything is broken up into small files with very specific, very focused functionality. You don't have to wade through thousands of lines of code to find what you're looking for like you do with ci::geom.

ci::geom really should be in its own folder rather than single file, and ideally sub-folders as well since there is the core framework, there are specific geometry types, there are computational classes, transformations, etc. If this were broken up into small files and sub-folders, it would be easy to add Normals for instance to complement the existing Tangents computation as an example.

If we only got Cinder broken up into small files and organized into sub-folders, I think that would be a big win.

[paulhoux]

I believe I am to blame for the monolithic nature of the ci::geom namespace. It began as a small collection of primitives, but quickly grew into the enormous beast it is today. I would not be against splitting it up into separate files. I would also refactor a bunch of code, since there is a lot of functionality in there that is very similar and 'almost copy-pasted'. For example, to draw a sphere you need to know how to draw a circle. So the sphere and circle share a lot of functionality that could potentially be moved to helper functions. But I feel like I am derailing the discussion, so I'll stop here.

[richardeakin]

I haven't had time to follow this thread in detail, but a couple thoughts from my end:

• re separate libraries for things like geom, signals: to me this isn't necessary on that level, if you want to use just a few of cinder's classes, then add those files directly to your project. Creating separate libraries would add a ton of maintenance work for little gain in practical situations.
• re all of ci::geom in a single file: I don't see the problem with this at all and actually prefer it to separate files per class. It is grouping related code, yet mostly independent from other parts of the library, into a single translation unit. If you any modern day editor it is easy enough to navigate to the class you want within GeomIo.h / cpp. If we were to break it out in any way, I would think we could separate it into geom::Sources and geom::Modifiers.