This example tries to show how to handle validate and compatible_packages features together.
This is a tiny scenario, but typically dependency graphs will be much bigger and information about libraries won't be as accessible as here.
Disclaimer.- This is a very opinionated dramatization using just two recipes, but it is based on a real story where I played the main character 😅
Alternative 1: recipes 'app' and 'library'
'library' recipe
In order to build your app you know you need library/1.0 and it is available as a Conan package (maybe in ConanCenter, maybe in your own servers) and you need to build it from sources:
conan export library/conanfile.py library/1.0@ # We need this here, it is not in any server
conan install library/1.0@ --build=library
If your compiler doesn't activate C++11 by default, we get a nice error:
ERROR: library/1.0: Invalid configuration: Current cppstd (gnu98) is lower than the required C++ standard (11).
and with a little knowledge of Conan we know how to fix it:
conan install library/1.0@ --build=library -s compiler.cppstd=11
...
Installing package: library/1.0
Requirements
library/1.0 from local cache - Cache
Packages
library/1.0:INVALID - Invalid
Installing (downloading, building) binaries...
ERROR: There are invalid packages (packages that cannot exist for this configuration):
library/1.0: Invalid ID: Current cppstd (11) is lower than the required C++ standard (14).
Opps! (We need a better error message here) Again a nice message that tells us we need C++14 to build the library. Easy to fix!
Now we want to use that library in our app. We forgot about the previous steps, maybe our app is upgrading from library/0.1 to library/1.0, maybe different teams take care of the library and the application:
conan export app/conanfile.py app/1.0@ # We need this here, it is not in any server
conan install app/1.0@ --build=app
...
ERROR: library/1.0: Invalid configuration: Current cppstd (gnu98) is lower than the required C++ standard (11).
Conan shows a nice message telling us that using library/1.0 requires at least C++11. app requires plain C++ but the new requirement needs C++11 to be used (API uses C++11 features):
conan install app/1.0@ --build=app -s compiler.cppstd=11
...
library/1.0: Main binary package 'INVALID' missing. Using compatible package 'ccfcfe63664926af39fa1a9f97d2f0dcf252dbf6'
...
app/1.0: Created
It's been easy, we have our application, all the binaries have been generated with the proper profile and can be identified. The binary we are using for library/1.0 belongs to compiler.cppstd=14 and the binary for app/1.0 is being built using a profile with compiler.cppstd=11. This can be important metadata associated with the binaries.
More importantly, this is quite straightforward from the POV of the user.
Alternative 2: recipes 'app' and 'hidden'
'hidden' recipe
The recipe for hidden package contains the typical lines CMAKE_CXX_STANDARD in the CMakeLists.txt file of the project, they are hidden to the consumer of the library. So we, Conan users, run the commands and we are able to compile hidden at the first shot:
conan export hidden/conanfile.py hidden/1.0@ # We need this here, it is not in any server
conan install hidden/1.0@ --build=hidden
This works because our compiler support C++14, otherwise, we would get the typical CMake error at build time. We know the error (not a template metaprogramming one, LOL).
Here we are already getting into troubles, we have generated a binary that uses C++14 but it is associated with the default profile that is using (at least my apple-clang) C++98... but other calls like:
We now go to the other team of developers, they retrieve the binary for hidden/1.0, the default profile works and they know the default for their compiler is C++98, their app doesn't use modern features, they can integrate this library. Let's go!
I get build errors! As an app developer I'm confused, this profile was suitable for the hidden/1.0@ package, but now it fails to build with of C++11 errors... is it my error? Am I missing something?
At least this is a small recipe, I can have a look at the errors, at the sources and at the recipe and I realize that I need C++11 because the API contains some auto declarations. Good, let's build the lib:
conan install app/1.0@ --build=app -o app:use_hidden=True -s compiler.cppstd=11
...
app/1.0: Forced build from source
Installing package: app/1.0
Requirements
app/1.0 from local cache - Cache
hidden/1.0 from local cache - Cache
Packages
app/1.0:f521448e6e11249788f58876ccd50841ecd1a160 - Build
hidden/1.0:INVALID - Invalid
Installing (downloading, building) binaries...
ERROR: There are invalid packages (packages that cannot exist for this configuration):
hidden/1.0: Invalid ID: Current cppstd (11) is lower than the required C++ standard (14).
WTF! I had a binary, now I need to build from sources the package hidden/1.0 because the new profile with compiler.cppstd=11 computes a new package ID... even more, it tells me that I need C++14, not only C++11. Ok, at least it is easy to fix:
conan install app/1.0@ --build=app -o app:use_hidden=True -s compiler.cppstd=14
...
ERROR: Missing prebuilt package for 'hidden/1.0'
Try to build from sources with '--build=hidden'
Use 'conan search <reference> --table table.html'
Or read 'http://docs.conan.io/en/latest/faq/troubleshooting.html#error-missing-prebuilt-package'
Why do I really prefer the first approach, where the C++ standard is managed by Conan and it doesn't appear in CMakeLists.txt at all:
Information is closer to the user, information is available when building the graph, not via an error from the build-system. This can be very important if the graph becomes big and complex, Conan can report the error after building the graph, no need to start building things to realize we need to change the compiler.cppstd value and start building again.
Metadata Binaries can be associated with the profile used to create them.
If the profile doesn't declare compiler.cppstd we know the binary uses whatever the default is for that compiler.
The C++ standard from the profile is the one used in the binary, it is not overridden inside.
This can be very important when using experimental features from future C++ standards that are not stable yet.
Better experience regarding ConanCenter packages. The binary available for hidden/1.0@ is not usable at all, we need a profile with compiler.cppstd=11 to be able to consume that package and this extra item in the profile modifies the packageID and it forces us to build from sources!!
Advantages of the second approach:
we don't need to patch every CMakeLists.txt file to remove the set(CXX_STANDARD XX) declaration because I know it is everywhere.
outside Conan, you don't need to pass the C++ standard in the command line in order to build the library using plain CMake commands.
IMHO, libraries will keep doing (2), but our recipes in ConanCenter (as soon as we start iterating configuration with explicit compiler.cppstd values) should remove those lines from CMakeLists.txt files.
This example tries to show how to handle
validateandcompatible_packagesfeatures together.This is a tiny scenario, but typically dependency graphs will be much bigger and information about libraries won't be as accessible as here.
Disclaimer.- This is a very opinionated dramatization using just two recipes, but it is based on a real story where I played the main character 😅
Alternative 1: recipes 'app' and 'library'
'library' recipe
In order to build your
appyou know you needlibrary/1.0and it is available as a Conan package (maybe in ConanCenter, maybe in your own servers) and you need to build it from sources:If your compiler doesn't activate C++11 by default, we get a nice error:
and with a little knowledge of Conan we know how to fix it:
Opps! (We need a better error message here) Again a nice message that tells us we need C++14 to build the library. Easy to fix!
We have our binary available now.
'app' recipe
Now we want to use that library in our
app. We forgot about the previous steps, maybe ourappis upgrading fromlibrary/0.1tolibrary/1.0, maybe different teams take care of the library and the application:Conan shows a nice message telling us that using
library/1.0requires at least C++11.apprequires plain C++ but the new requirement needs C++11 to be used (API uses C++11 features):It's been easy, we have our application, all the binaries have been generated with the proper profile and can be identified. The binary we are using for
library/1.0belongs tocompiler.cppstd=14and the binary forapp/1.0is being built using a profile withcompiler.cppstd=11. This can be important metadata associated with the binaries.More importantly, this is quite straightforward from the POV of the user.
Alternative 2: recipes 'app' and 'hidden'
'hidden' recipe
The recipe for
hiddenpackage contains the typical linesCMAKE_CXX_STANDARDin theCMakeLists.txtfile of the project, they are hidden to the consumer of the library. So we, Conan users, run the commands and we are able to compilehiddenat the first shot:This works because our compiler support C++14, otherwise, we would get the typical CMake error at build time. We know the error (not a template metaprogramming one, LOL).
Here we are already getting into troubles, we have generated a binary that uses C++14 but it is associated with the default profile that is using (at least my
apple-clang) C++98... but other calls like:will succeed too, they will use C++14 even though I'm requesting C++11. This can be a problem from the ABI perspective in some scenarios.
Metadata associated with these binaries will be wrong.
'app' recipe (using hidden requirement)
We now go to the other team of developers, they retrieve the binary for
hidden/1.0, the default profile works and they know the default for their compiler is C++98, theirappdoesn't use modern features, they can integrate this library. Let's go!I get build errors! As an
appdeveloper I'm confused, this profile was suitable for thehidden/1.0@package, but now it fails to build with of C++11 errors... is it my error? Am I missing something?At least this is a small recipe, I can have a look at the errors, at the sources and at the recipe and I realize that I need C++11 because the API contains some
autodeclarations. Good, let's build the lib:WTF! I had a binary, now I need to build from sources the package
hidden/1.0because the new profile withcompiler.cppstd=11computes a new package ID... even more, it tells me that I need C++14, not only C++11. Ok, at least it is easy to fix:Oh! Of course, let's use
--build=missing:...but I built my
appusing C++14, and I wanted it just with C++11, one more compilation:And I'm finally there.
Why do I really prefer the first approach, where the C++ standard is managed by Conan and it doesn't appear in
CMakeLists.txtat all:Information is closer to the user, information is available when building the graph, not via an error from the build-system. This can be very important if the graph becomes big and complex, Conan can report the error after building the graph, no need to start building things to realize we need to change the
compiler.cppstdvalue and start building again.Metadata Binaries can be associated with the profile used to create them.
compiler.cppstdwe know the binary uses whatever the default is for that compiler.This can be very important when using experimental features from future C++ standards that are not stable yet.
Better experience regarding ConanCenter packages. The binary available for
hidden/1.0@is not usable at all, we need a profile withcompiler.cppstd=11to be able to consume that package and this extra item in the profile modifies the packageID and it forces us to build from sources!!Advantages of the second approach:
CMakeLists.txtfile to remove theset(CXX_STANDARD XX)declaration because I know it is everywhere.IMHO, libraries will keep doing (2), but our recipes in ConanCenter (as soon as we start iterating configuration with explicit
compiler.cppstdvalues) should remove those lines fromCMakeLists.txtfiles.