search

conan-io / conan-extensions

Some extra Conan commands for different purposes, like artifactory tasks, conan-center-index, etc
MIT License
30 stars 27 forks source link

Set the Artifact.properties using the CI/CD building process (TeamCity) #67

Open smileNika opened 1 year ago

smileNika commented 1 year ago

Hi team,

I Have not found a solution for my request.

1) I have a task - I need to save some properties about git_commit / build_number _CI_CD/ HW_version in Artifactory. I found artifact.properties** file in .conan cache folder which is possible to edit only locally. And after editing artifact.properties file I could upload packages manually into Artifactory repository. That is ok, and working fine. But how to be with CI/CD building, when packages uploaded automatically ? How to edit this file in this case (like in server side)? Maybe, there are any other possibilities to save the props in packages revision?

2) One more question, how to read this properties? I mean not only in Artifactory GUI, but in command line. It is needed to check firstly which package to install based on these props values. For example to see all recipe revisions or package ID, or package revisions I could use conan search command. But what about properties? Are there any commands to represent it in terminal?

memsharded commented 1 year ago

Hi @smileNika

This is the conan-extensions repo, which is exclusively for Conan 2.0. The mechanism of the artifact.properties file is a Conan 1.X, that has been removed in Conan 2.0. In any case, this file shouldn't be edited, it is there mostly for the old way to generate Artifactory BuildInfo (that no longer works)

For Conan 2.0 the command is the properties command in https://github.com/conan-io/conan-extensions/tree/main/extensions/commands/art

smileNika commented 1 year ago

@memsharded Hi. Thank you for your reply. In my task I have to use conan 1.X (1.53 1.54). Then what is the correct way to store all BuildInfo?

How the properties info usually save in Artifactory in 1.X versions of conan? Because as I understand from your words - artifact.properties file is just an output log file which represent what we could manually fulfill in Artifatory. But in a CI continuous building process it is not good idea open Artifactory and try to paste each important peace of information by myself. There should be an automation way continuously store updated building info for each package and represent this info before installation.

One more question: I have seen in 2.0 documentation it is bad way to store multiple package revision for one recipe revision. But in a 1.X - there are lot of discussion regarding this topic. How to idetify PREV and RREV before installation. So why did you advice not to store few PREV in one RREV? and are there any mechanism how to prevent it towards good way of package storing?

memsharded commented 1 year ago

Thank you for your reply. In my task I have to use conan 1.X (1.53 1.54). Then what is the correct way to store all BuildInfo?

I am afraid that the BuildInfo doesn't work greatly with Conan 1.X (it never worked greatly, it might be generated, but it has some structural limitations), and at this stage this won't be a priority. The docs how to do it are in https://docs.conan.io/1/howtos/generic_ci_artifactory.html

How the properties info usually save in Artifactory in 1.X versions of conan?

In Conan 1.X, properties had to be managed via Artifactory API or the JFrog CLI, Conan never had a mechanism to handle Artifactory properties.

One more question: I have seen in 2.0 documentation it is bad way to store multiple package revision for one recipe revision. But in a 1.X - there are lot of discussion regarding this topic. How to idetify PREV and RREV before installation. So why did you advice not to store few PREV in one RREV? and are there any mechanism how to prevent it towards good way of package storing?

To clarify, multiple package-revisions PREV for the same package-id of the same recipe-revision. When revisions were introduced in Conan 1.X, we also tried to cover the ill-defined case of users modifying the environment without modeling the changes at all, and Conan ignoring those changes. That could be captured/fixed by the package revisions.

Then we learned that this is a bad approach, leading to many other problems. The right approach is modeling things correctly (like adding new settings.yml to model custom binary characteristics or changes in the environment).

So having more than 1 package revision can be avoided in 2.0 with:

smileNika commented 1 year ago

okay. Understod about JFrog - then I need to learn more about its API or CLI aproach. The question still - is how to check this props when I want to install specific Recipe revision to decide which one revision I need to install.

about package revisions ----> are there any possibilities to handle it (1RREV : 1PREV) in conan 1.53 version? Could I restrict explicitly creation multiple packages for 1 recipe revision?

memsharded commented 1 year ago

The question still - is how to check this props when I want to install specific Recipe revision to decide which one revision I need to install.

Conan doesn't check properties to decide to install one revision or the other. The revision used by Conan is resolved with:

For Conan, the Artifactory properties does not exist in the dependency resolution, so they cannot be used to decide about a specific revision. And most likely they shouldn't, this would be aligned with other package managers for other languages. The version (and revision, which is part of the version) resolution always follow rules based on it.

What you can do in the JFrog platform is use the infrastructure, for example you can tag with certain properties some specific revisions. Then you can use those properties to run a copy of the artifacts with some specific value of that property to another repository. Finally you can point Conan (or other language package manager) to that repository you copied the packages, and Conan will resolve those, because those are the only ones in that repository.

about package revisions ----> are there any possibilities to handle it (1RREV : 1PREV) in conan 1.53 version? Could I restrict explicitly creation multiple packages for 1 recipe revision?

Yes, as long as you don't run conan create indiscriminately (you can check with info the existence of binaries, or you can use --build=missing), and you don't force the build of packages with -build, --build=* or anything like that, having more than 1 prev can be avoided (Recall, the relation is (1rrev-1package_id: 1prev), you cannot drop the package_id from there, it is critical for the model to work). You might need to change the default package_id_modes, as they are too relaxed in Conan 1.X by default, to guarantee correct re-build from source when it is necessary.