[question] Cross-building on Windows (build-profile) for NI LinuxRT (host-profile) and using of gRPC

[ThomasDBecker]

What is your question?

Hello,

I'm relatively new, but I have to say that I've become a bit of a fan of Conan in a short time.

What I'm trying

Attached is a CMake project that is intended to build a test application on Windows for Windows and NI-LinuxRT. The test application uses a shared library (DLL or SO), which includes a static library. gRPC should be used in the static library.

Windows -> Windows

First I got the following error:

Until I changed TLS_VERIFY from ON to OFF in the download_archive.cmake file.

Question 1: I know this doesn't have anything to do with CONAN directly, but what would you suggest as best praxis here?

After this change I can compile and debug the binaries in debug on Windows for Windows.

Fine

Windows -> NI LinuxRT

The NI LinuxRT Toolchain can be downloaded from here: GNU C & C++ Compile Tools x64

The toolchain contains:

• GNU C/C++ Compiler (Windows executable (exe) which can generate binaries for LinuxRT (o, so, a, …)
• The compiler provides a --sysroot= flag to specify a logical root directory for libraries and headers for proper linking and includes
• Debugger: The GNU Debugger (gdb) is a Windows exe which can be connect to a remotely running GNU Debugger Server (gdbserver) on LinuxRT

It is important that the following environment variables are set correctly. I do this in conanfile.py.

First I got an error message that I couldn't really do anything with: flex/2.6.4: Invalid: Flex package is not compatible with Windows. Consider using winflexbison instead

Until I changed the following in the file C:\Users....conan2\p\libse592dd5dc2df7d\e\conanfile.py:

Question 2: Is there perhaps more information here about why this change has to be made and is this change even correct? What is the general recommendation here?

Question 3: Unfortunately, I'm now getting the following error message, which I can't quite identify. It would be great if someone could help me.

TestApplication02.zip

Question 4 What is the CONAN equivalent of VCPKG's features?

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[memsharded]

Hi @ThomasDBecker

Thanks for your question and thanks for your kind words!

Question 1: I know this doesn't have anything to do with CONAN directly, but what would you suggest as best praxis here?

Maybe this is happening because you are running in a corporate environment? We have seen these issues in the past and most of the times is because the corporate network/firewall/vpn/proxy/zscaler... is not listing as allowed some certificates or domains. The recommendation is to request IT to unblock the URLs.

If it is not this issue, there is relatively little that Conan can do, because as you can see this is something that is hardcoded in the library CMakeLists.txt. Maybe as a best practice, doing an external patch instead is better, as it helps identify and maintain the external changes and can help also in future versions that might change this code.

(I will follow up the other questions in different comments, to help tracking and referencing in the conversation)

[memsharded]

Question 2: Is there perhaps more information here about why this change has to be made and is this change even correct? What is the general recommendation here?

In Question 2 there are 2 different aspects.

The first one is that I think that it could be easier to set the environment. Recipes don't need to be modified to add env-vars, as they can be added from the [buildenv] section of a profile or they can be defined in another tool_requires recipe that can be injected, that would be more convenient as it can be generalized for all recipes without modifying them. If you want more detail about this topic, I think the best would be to create a dedicated new issue for it.

Regarding the change in the grpc library for flex/winflexbison let me have a look. What I can say is that modifying things in the Conan cache is not allowed, as it corrupt packages. The best way to customize recipes from conan center is to fork conan-center-index github repo, do the changes in the fork and conan create . --version=version in the recipe folder.

UPDATE: I'd need a bit more detail of the grpc recipe, because the one from ConanCenter got the build_requirements():

 def build_requirements(self):
        if not self._is_legacy_one_profile:
            self.tool_requires("protobuf/<host_version>")
        if cross_building(self):
            # when cross compiling we need pre compiled grpc plugins for protoc
            self.tool_requires(f"grpc/{self.version}")

So I'd need to understand better that recipe for grpc, where is it coming from, etc.

[ThomasDBecker]

Hi @memsharded ,

thank you for your fast and helpfull answers.

In terms of Question 1 (TLS_VERIFY from ON to OFF) Yes, my project is running in a corporate environment and I almost suspected that. Thank you for your confirmation.

In terms of Question 2 (setting of environment variables in a recipe)

The first one is that I think that it could be easier to set the environment. Recipes don't need to be modified to add env-vars, as they can be added from the [buildenv] section ...

I chose this way to automate the settings of the environment variables and not to assign fixed values ​​for environment variables in the profiles.

I don't really understand the second part of the answer at the moment, but that's my fault. When I have time, I'll look into it in more detail and possibly go back to your recommendation. But my priority here isn't that high at the moment.

In terms of Question 2 (flex/winflexbison)

What I can say is that modifying things in the Conan cache is not allowed, as it corrupt packages

Makes total sense and I thought that too, I just wanted to try and see if I could get something running

The best way to customize recipes from conan center is to fork conan-center-index github repo, do the changes in the fork and conan create . --version=version in the recipe folder

Thanks for the tip

UPDATE: I'd need a bit more detail of the grpc recipe, because the one from ConanCenter got the build_requirements()...So I'd need to understand better that recipe for grpc, where is it coming from, etc.

Thanks for the update. Yes, sure, makes sense.

[memsharded]

Thanks for your feedback!

I chose this way to automate the settings of the environment variables and not to assign fixed values ​​for environment variables in the profiles.

Profiles can also be parameterized with jinja syntax. I didn't fully understand where you are defining those env-vars, it would be good to know it better. Because in any case, having to modify recipes to add the env-vars doesn't sound great, I think it shouldn't be necessary for most cases.

I don't really understand the second part of the answer at the moment, but that's my fault. When I have time, I'll look into it in more detail and possibly go back to your recommendation. But my priority here isn't that high at the moment.

Don't worry, no rush. You can always later open dedicated new tickets for any specific issue you want.

[memsharded]

For question 4:

What is the CONAN equivalent of VCPKG's features?

The equivalent are Conan options. Recipes in ConanCenter for example often define options, such as with_feature1 or without_feature1, depending if the package has them as opt-in or opt-out. Activating and deactivating them is also done in profiles with:

[options]
mypkg/*:mypkg_option=value

Note the mypkg/* is a pattern to apply the options. It is common for example to want all dependencies as shared libraries, so *:shared=True, or there might be situations when the options are to be applied only to a specific version, so mypkg/1.0:mypkg_option=value could be used.

These can be defined also in command line with -o "*:shared=True", and they compose with existing profiles with higher priority, so -pr myprofile -o "*:shared=True" will use shared=True even if the profile defined *:shared=False.

[memsharded]

For Question 3:

Question 3: Unfortunately, I'm now getting the following error message, which I can't quite identify. It would be great if someone could help me.

This seems very related to the above issue: it is a certificate issue that is being excluded by your network configuration.

In this case, as this is a Conan download, it might be possible to force not using a certificate with:

conan install ...  -c tools.files.download:verify=False

Please try that and let us know.

If you are curious, you can get the whole list of configurations with conan config list

Disabling the certificate verification would be a workaround, the right solution would be to have your IT to also allow this domain/certificate.

[ThomasDBecker]

Hi @memsharded ,

thank you again for your fast and helpful answers.

I didn't fully understand where you are defining those env-vars, it would be good to know it better

GNU C & C++ Compile Tools x64 comes with a batch file. I am currently reading it from conanfile.py, adjusting the environment variables and then setting it with the following piece of code:

            tools = NiGnuCCppCompileToolsX64()
            envVars = tools.getEnvVars(os.path.join(self.recipe_folder, "toolchain-23Q4"))

            buildenv = CONAN2_TOOLS_ENV.VirtualBuildEnv(self) # https://docs.conan.io/2/reference/tools/cmake/cmaketoolchain.html#presets-build-environment-presets-run-environment
            for variable in envVars:
                buildenv.environment().define(variable, envVars[variable])
                printYellow("=" * (self.indent + 1) + f"{variable} = >>{envVars[variable]}<<")            
            buildenv.generate()

            cmakeToolchain.presets_build_environment = buildenv.environment()

The class NiGnuCCppCompileToolsX64 is self-built, but as I said, this topic is not that important. It was more of an experiment.

Because in any case, having to modify recipes to add the env-vars doesn't sound great, I think it shouldn't be necessary for most cases

I will definitely take this advice into consideration.

The equivalent are Conan options...

Very interesting and very helpful. I will look at this chapter in more detail in the next few days.

conan install ... -c tools.files.download:verify=False Please try that and let us know

Yes, it works for me.

Disabling the certificate verification would be a workaround, the right solution would be to have your IT to also allow this domain/certificate

Of course. I can only agree. Good point.

[memsharded]

GNU C & C++ Compile Tools x64 comes with a batch file. I am currently reading it from conanfile.py, adjusting the environment variables and then setting it with the following piece of code:

Understood. As this is something that might be done and apply to multiple recipes, I think there could be some ways to do it that would avoid repetition:

• Conan profiles are jinja2 templates, they support powerful syntax, it might be possible to read the .bat file from the profile, parse it and then define it as [buildenv] section in the profile.
• You can put the logic that loads the file and defines self.buildenv_info inside the package_info() method of a package, lets call it mytoolchain_envvars/1.0 then, these variables will be automatically injected just by adding mytoolchain_envvars/1.0 to the tool_requires, which an in turn be automatically injected in profiles via [tool_requires]

Yes, it works for me.

Happy the verify=False conf helped.