Unknown command 'conan_build_info'

[GordonJess]

I want to run the conan_build_info command, but since conan is prepended to the command argument, it is not recognized:

Is there any other way to call this command (--v2)?

expected to call org.jfrog.hudson.pipeline.common.types.ConanClient.run but wound up catching runConanCommand; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/
[<job name>] $ cmd.exe /C "conan conan_build_info --v2 start <job name> <build number> && exit %%ERRORLEVEL%%"
ERROR: Unknown command 'conan_build_info'
'conan_build_info' is not a Conan command. See 'conan --help'.

[jgsogo]

Hi, @GordonJess

Right now the only way to do it is to call sh directly in your pipeline

sh "conan_build_info --v2 ...."

but we know that we should add this new command to the Artifactory plugin and use it by default to gather build information. And we will submit a PR with these changes asap.

---

Meanwhile I'm working with @czoido in a POC related to CIs using Conan and build info. You can find the toy project with all the dependency graph in this organization (https://github.com/demo-ci-conan), and a Jenkinsfile within each repository. You can see there how we are using conan_build_info right now to gather all the build information (https://github.com/demo-ci-conan/libA/blob/master/Jenkinsfile). We would love to hear your feedback, but be gentle, it is still a work-in-progress, some pieces are missing, there are some workarounds,... but eventually everything should be included in Conan and in this plugin.

We will write about it when we are happy enough with the implementation

[jgsogo]

After using the new conan_build_info --v2 command and some issues/fixes that should be released with Conan 1.21, we can start to think about implementing the new functionality in this plugin.

It will require the plugin to know the installed Conan version (let's make it a standalone function, as it can be useful somewhere else) to see if the new command is available, and then implement the new behavior but keep the old behavior as the default one (add a v2 argument to activate the new behavior). After a couple of Conan releases, we should probably change it to be the default behavior.

@czoido, as you are more into the conan_build_info command, can you summarize the steps that should be done in order to gather the build info and publish it?

@eyalbe4, can you change the issue title to match something like: "Implement new 'conan_build_info' command" and tag the issue accordingly? (We will try to work on it soon).

Thanks!

[czoido]

Yes @jgsogo, as you said we are testing the conan_build_info --v2 right now and solving some issues. The procedure to use the conan_build_info --v2 command should begin by the call to start so that the files are uploaded to Artifactory using the correct build_name and build_number:

$ conan_build_info --v2 start MyBuildName 42

Then, after the creation of packages and upload to Artifactory the build info is generated using the Conan lockfiles and passing the credentials for the Artifactory repo:

$ conan_build_info --v2 create buildinfo.json --lockfile conan.lock --user admin --password password

After the build info is created it can be published to Artifactory passing the remote credentials again:

$ conan_build_info --v2 publish buildinfo.json --url http://localhost:8081/artifactory --user admin --password password

If you don't want to continue associating the build_name and build_number you specified in the start subcommand with the files you upload to the remote you have to use the stop subcommand:

$ conan_build_info --v2 stop

Also, the update subcommand is very useful to merge the build info generated by different processes in the same build:

$ conan_build_info --v2 update buildinfo1.json buildinfo2.json --output-file mergedbuildinfo.json

Links to documentation: https://docs.conan.io/en/latest/reference/commands/misc/conan_build_info.html https://docs.conan.io/en/latest/howtos/generic_ci_artifactory.html

Also, a link to an organization we are using to test possible CI/CD flows using Conan features such as lockfiles, package id modes and conan build info: https://github.com/demo-ci-conan

[jgsogo]

(Sharing some ideas, pseudocode)

From the plugin perspective, the usage should be as transparent as possible. For this proposal I need to introduce some new methods to the ConanClient class that will perform just the Conan action they are referring to:

• Commands that collect information: conan.install and conan.create. These commands will store the profile used and the package references being created.
• Commands that create the build information: conan.upload. This command will upload packages, given the package references uploaded, it can retrieve from the previous ones the profile used, recompute the graphlock and aggregate them it a BuildInfo object.

This will cover several use cases:

• Regular workflow: we use a job to create a package

  // Running in a node
  def server = Artifactory.server artifactory_name
  def client = Artifactory.newConanClient()
  def remoteName = client.remote.add server: server, repo: artifactory_repo
  
  client.create(reference: "name/version", profile: "myprofile")
  
  def buildInfo = client.upload(reference: "*", remote: remoteName)
  server.publishBuildInfo buildInfo

• Several configurations: the client.upload is able to merge existing build information.

  // Running in a node
  def server = Artifactory.server artifactory_name
  def client = Artifactory.newConanClient()
  def remoteName = client.remote.add server: server, repo: artifactory_repo
  
  client.create(reference: "name/version", profile: "pr1")
  def buildInfo = client.upload(reference: "*", remote: remoteName)
  
  client.create(reference: "name/version", profile: "pr2")
  def buildInfo = client.upload(reference: "*", remote: remoteName, buildInfo: buildInfo)
  
  server.publishBuildInfo buildInfo

  If running in different jobs, the buildInfo object should be stashed and retrieved in the orchestrator to be merged with the ones coming from all the nodes. This is TBD, but it should fit with this implementation: the plugin shall gather all the buildInfos and merge them.

• Moving packages around:

  // Running in a node
  def server = Artifactory.server artifactory_name
  def client = Artifactory.newConanClient()
  def remoteName = client.remote.add server: server, repo: artifactory_repo
  
  client.install(reference: "name/version", profile: "myprofile")
  def buildInfo = client.upload(reference: "name/version", remote: remoteName)
  server.publishBuildInfo buildInfo