[question] What is Conan 2's story for deploying cross compiled binaries to a remote machine?

[ChrisThrasher]

What is your question?

I'm using Conan 2.0.2 to cross compile a collection of 1st and 3rd party code. The target hardware is not hardware on which we can directly do development, hence the requirement to cross compile everything. Conan 2 has a clear story for how cross compilation works but I'm struggling to figure out the best way to get those compiled shared libraries and executables onto our target hardware.

I had a talk with @prince-chrismc after which we determined that Deployers are not the right solution. If not Deployers then how do you recommend I copy these binaries to the target hardware? Am I best off doing something like using rsync in a shell script to copy the required files over or do more sophisticated solutions exist?

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[memsharded]

Hi @ChrisThrasher

Thanks for your question

I had a talk with @prince-chrismc after which we determined that Deployers are not the right solution. If not Deployers then how do you recommend I copy these binaries to the target hardware?

I'd like to understand better why not. Deployers are quite generic tool, and for the purpose of extracting files from Conan packages and doing something with them, it is exactly its purpose. If you could please elaborate a bit more.

[ChrisThrasher]

The original issue summarizes it. I'm cross compiling a lot of code and need to put those shared libraries and executables on some specialized target hardware which can run it. I'm hoping that Conan provides a convenient way of doing that.

[memsharded]

The original issue summarizes it. I'm cross compiling a lot of code and need to put those shared libraries and executables on some specialized target hardware which can run it. I'm hoping that Conan provides a convenient way of doing that.

I am not sure if I follow. The first step is to extract the executables, shared libraries from the Conan packages, and create the format that is going to be deployed on the target. Lets say that it is a folder with everything inside. That is perfectly done with the deployers, it is exactly its purpose.

From there, I am afraid there is little Conan can do (as built-in), if Conan cannot run in the target system, the deployment of a folder from your current machine when Conan is running, to a random target machine depends on a myriad of things, so that would be something quite outside of Conan scope? Lets say that it is a "ftp" directly to the target (I hope not 😅 ) or a rsync, or uploading the tgz to a server, and then the target system can pull it from the server. There could be some possibilities this task could be automated with Conan extensions:

• A deployer. You can run as many deployers as you want, and they run in order. So you can run a deployer that will collect the files from the Conan packages and put them in a folder, then another deployer could rsync that folder to the target. Maybe the deployer could directly deploy to the target, but I think I prefer to decouple it into 2 different depoyers
• A deployer + a custom command: A conan install --deploy=xxxx extracts the artifacts from Conan packages, then a custom command conan mycmds:deploy myfolder --target=url_to_target will manage to do the rsync to the target
• A custom command like conan mycmds:install_and_deploy /path/to/conanfile -pr=target_profile --target=url_to_target this will implement an iteration internally to the command similar to what the deployers are doing, it will probably be almost identical code to the one in the deployer.

[ChrisThrasher]

I think you're following. Your answer makes sense.

I suppose since deployers cannot accept additional arguments like a hostname, they're not suitable for directly deploying to a remote machine since that remote machine is going to vary over time and we don't want to be hardcoding a hostname into our deployer script.

In that case I think I'll go forward using a custom deployer to collect all the relevant runtime assets and put them into a folder which can then be copied to the target hardware through some means unrelated to Conan. Are there any conveniences conan provides for collecting all those shared libs and executables or am I on my own to traverse the build tree(s) to find them all?

[prince-chrismc]

if Conan cannot run in the target system, the deployment of a folder from your current machine when Conan is running, to a random target machine depends on a myriad of things, so that would be something quite outside of Conan scope?

I think is my expertise ran out, it was not clear if they was supportable or not :) Trying to deploy with Conan requires it be able to run on the target machine, but I think that acceptable answer to the deploy cross-building applications with Conan.

A: Make a package bundle and solve the "copy" through another machoism.

Are there any conveniences conan provides for collecting all those shared libs and executables or am I on my own to traverse the build tree(s) to find them all?

Today you are probably on your own, however I can share https://github.com/prince-chrismc/conan-whatsnew2/blob/main/deployers/.conan/extensions/deploy/runtime_zip_deploy.py

Which I will try to push into https://github.com/conan-io/conan-extensions which will the place we share generally convient tools like these :)

[ChrisThrasher]

I'm still confused about one thing. The implication of running conan install is that the dependencies of a given package get deployed. That does not mean that the package you're actually building gets deployed. I need a way to deploy the dependencies and the compiled artifacts of the package that is using all of those dependencies. Does that make sense?

[prince-chrismc]

You can conan install conanfile.py or conan install --requires name/version. Both give you the dependencies graph, how the later has a "virtual root" where the requirement (i.e the final application) it complete and you can install it

https://github.com/prince-chrismc/conan-whatsnew2/blob/5cb200e744ebee8280e981e402b9e945cafd9c2e/build.py#L82 for example

You can not --deploy a package which has not been built. the assumption is that conan create has already ran and populated the local cache at least

[memsharded]

I suppose since deployers cannot accept additional arguments like a hostname, they're not suitable for directly deploying to a remote machine since that remote machine is going to vary over time and we don't want to be hardcoding a hostname into our deployer script.

Yes, that is true, deployers cannot receive custom args on the fly (a deployer can always take parameters from a file in disk, or from env-vars)

In that case I think I'll go forward using a custom deployer to collect all the relevant runtime assets and put them into a folder which can then be copied to the target hardware through some means unrelated to Conan. Are there any conveniences conan provides for collecting all those shared libs and executables or am I on my own to traverse the build tree(s) to find them all?

Yes, make sense. When I was suggesting Conan custom commands is because those can be convenient:

• can be redistributed together with other things (profiles, other commands) with conan config install
• A unified syntax, familiar to developers That can be handy for developers, but sure other mechanism sounds perfectly good.