search

giuspen / cherrytree

cherrytree
https://www.giuspen.net/cherrytree/
Other
3.36k stars 459 forks source link

Use Conan for dependency management #996

Open ForeverRainbow opened 4 years ago

ForeverRainbow commented 4 years ago

Hi I mentioned this a while back in #444 but it kind of got lost. Here I am going to lay out what I think the benefits to using conan would be, some of the potential problems I can see, what Conan actually is, and the packages currently supported.

Here is a link to the Conan docs which I recommend reading if you do not know what Conan is (they are very good). I briefly explain what it is here but not in detail.

Packages for dependencies

There are currently packages for most of cherrytree's dependencies, either in the conan-center or otherwise

In the conan-center (doesn't require any additional configuration)

conan-cpputest provides a recipe for cpputest however it does not seem to be hosted so will require some setup.

gtk3mm does not have a conan package that I could find but is installable through system package managers. Additionally Conan may allow the (non-trivial) building gtk3mm on windows in a reusable and not painful to set up way.

What is it?

Some background for people who don't know what conan is. Conan is an open source package manager, like vcpkg (but not made by microsoft). I have some experience with it and have used it for several projects. It is similar to npm (js) or cabal (Haskell) or other package managers which exist for most "modern" languages, it works with a recipe system where packages specify their dependencies and build system (see this list of built-in build helpers). Consumers do the same (although since they are not distributed the build instructions are optional) and can use one of the built-in generators to generate dependency descriptions for many built systems (or write your own). There is a pkg-config and 3 cmake generators (of which the cmake_find_package one makes conan completely transparent up to a point). Recipes can be in either .ini format for simple dependencies or a python script for complex ones (e.g changing dependencies depending on build platform).

Once a recipe for a package has been created installing dependencies is as simple as typing conan install <path to recipe folder> and the dependency files will be generated in your current directory (e.g with the pkg-config generated there will be a bunch of .pc files), if the package does not have a pre-built binary available for the current toolchain then you can build it from source by specifying -b missing. If build instructions are available then you can run conan build <path to recipe folder> after install and it will build the package (you can specify environment variables and such as required).

Why would it be useful?

Currently cherrytree dependencies are either installed through the system package manager or included right into cherrytree's source. Including dependencies into the source increases clean build time (although that is not such an issue) and also clutters the commit history while making it very difficult to update or change versions. The system package manager works but makes cross-compiling difficult to impossible if your system does not have the required packages (just try getting mingw-gtk3mmsourceview on fedora, please let me know if you can). Additionally if a dependency doesn't use CMake and doesn't provide system packages for common systems then it would be impossible (or at least hell) to integrate it into the build and would require another step in the build process for users.

Conan solves these issues by offering a single way to install packages across windows, mac and linux, as well as supporting cross-compilation (it has a concept of host and target systems and profiles allow changing of build environments, see the docs for more info on this). It also allows easy updating of embedded dependencies and version freezes for system packages (dependencies are specified as <name>/<version>[@channel], the <version> can be dynamic like [>=1.27] or fixed like 1.27). It is also build system antagonistic, a package built with meson can have find_package files, .pc files, .xcconfig files (xcode integration), whatever your fancy generated for it by Conan.

It also makes adding and removing dependencies much easier, for example boost has a modular conan package which allows only installing/building the parts actually needed.

Currently there are install instructions custom tailored to different linux distributions and different systems, by using Conan different system requirements could be specified in the recipe, making per-platform configuration transparent when building. The entire build process for a Conan package for example is this:

What are the drawbacks?

The most prominent drawback I can see is that it is another thing to maintain and knowledge of python and Conan would be required to add dependencies or fix issues.

Conan also does not have a package for gtk3mm in the conan-center (the central repository for packages) or bincrafters (a community of package creators which maintain most packages not in the conan-center). There are some packages available from individuals but we would probably either need to write our own (which I might be able to do) or utilise the SystemPackageTool class available in python script recipes to install it using the system package manager.

Also I have had difficulties in the past getting travis to play nice with it, since it requires a python dependency. As of yet I have not successfully gotten it to work (although there are instructions in the docs).

How would it be implemented?

My idea for how it would replace the current dependency system is that there would be a conanfile.py in /future with the recipe for cherrytree which would have spdlog, fmt, and 7zip, etc as requirements with cpputest as a build requirement. Then use SystemPackageTool to install the required package on a per-os basis. Then the install instructions would be:

I will create a simple recipe for cherrytree, if this sounds promising, to see how difficult it would actually be to move.

giuspen commented 4 years ago

@ForeverRainbow I'm quite happy with the build system we have at the moment, note about the dependencies that you listed:

ForeverRainbow commented 4 years ago

@giuspen Conan isn't a build system, CMake would still be used, it just manages dependencies (and can do it transparently so the current way would still work if needed). Didn't know 7zip was customized, but I listed fmt because it is a dependency of spdlog (although we are using the bundled fmt). cherrytree does dependend on sqlite3 though unless I am much mistaken as well as libxml2 (indirectly through libxml++).

I will look at the imports, have been a bit busy with other stuff recently :p.

giuspen commented 4 years ago

Thanks @ForeverRainbow ! If you consider also external libraries like sqlite3 and libxml2 then there is more to add, I thought you were considering only what we embed

ForeverRainbow commented 4 years ago

I successfully got this working earlier btw, it is basically just a translation of the install guide in /future that just installs those packages for the users system automatically (and uses a package for cpputest so it doesnt have to be built from source manually). I have it here: https://github.com/ForeverRainbow/cherrytree/blob/master/future/conanfile.py if anybody wants to check it out, its mostly just a proof of concept but it (should) work on a fresh system.

It gets the CppUTest package from my bintray currently but that is just a build of the cpputest package from the cpputest github so anyone can host it if they want or just package it locally.