[question] How to limit scope of replace_requires to specific recipe?

[Nekto89]

What is your question?

Hi,

I'm trying to use openssl with FIPS. This requires having openssl that depends on openssl. It became possible in conan recently. I currently have conanfile.py that is generated dynamically from CMake and it forces specific versions of all the packages that I need so that I don't depend on versions in recipes. How do I override hidden dependency of openssl/3.0.14? I've tried using [replace_requires] but I'm slightly confused by the output. I'm not sure what version is used in the end - there are "Replaced requires" and "Overrides" at the same time. I would like to have 3.0.14 everywhere except for openssl/3.0.14 itself. Can I do this through recipe instead of replace_requires?

[replace_requires]
openssl/3.0.9: openssl/3.0.9@user/channel
zlib-ng/*: zlib-ng/2.2.1@user/channel

Replaced requires
    zlib-ng/2.1.6: zlib-ng/2.2.1@user/channel#e3183aa8a73e8c1ee022af19f46edd3f
    openssl/[>=1.1 <4]: openssl/3.0.9@user/channel#ffb3f96cda0658e55369c4ece26c3292
    openssl/3.0.9: openssl/3.0.9@user/channel#ffb3f96cda0658e55369c4ece26c3292
    zlib-ng/2.2.1: zlib-ng/2.2.1@user/channel#e3183aa8a73e8c1ee022af19f46edd3f
    zlib-ng/2.1.3: zlib-ng/2.2.1@user/channel#e3183aa8a73e8c1ee022af19f46edd3f
Overrides
    zlib-ng/2.2.1@user/channel#e3183aa8a73e8c1ee022af19f46edd3f: [None, 'zlib-ng/2.2.1@user/channel#e3183aa8a73e8c1ee022af19f46edd3f']
    openssl/3.0.9@user/channel#ffb3f96cda0658e55369c4ece26c3292: [None, 'openssl/3.0.14@user/channel#ffb3f96cda0658e55369c4ece26c3292']

from conan import ConanFile
from conan.tools.cmake import CMakeDeps
from conan.tools.files import copy

class MyApp(ConanFile):
    name = "app"
    version = "1.0"
    settings = "os", "arch", "compiler", "build_type"

    def requirements(self):
        #there are lots of packages here, some of them depend on openssl, and some of them depend on zlib-ng
        self.requires("openssl/3.0.14@user/channel#ffb3f96cda0658e55369c4ece26c3292", force=True)
        self.requires("zlib-ng/2.2.1@user/channel#e3183aa8a73e8c1ee022af19f46edd3f", force=True)

    def generate(self):
        cmake_deps = CMakeDeps(self)
        cmake_deps.configuration = "Release"
        cmake_deps.generate()
        for dep in self.dependencies.values():
            if len(dep.cpp_info.bindirs):
                copy(self, "*.dll", dep.cpp_info.bindirs[0], r"C:/dev/bld/Binaries/Release")
            if len(dep.cpp_info.libdirs):
                copy(self, "*.dylib*", dep.cpp_info.libdirs[0], r"C:/dev/bld/Binaries/Release")
                copy(self, "*.so*", dep.cpp_info.libdirs[0], r"C:/dev/bld/Binaries/Release")

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[memsharded]

Some side suggestion:

• copy(self, "*.dll", dep.cpp_info.bindirs[0], r"C:/dev/bld/Binaries/Release") this seems an anti-pattern. Conan got the deploy() method to control what/when to copy things in folders outside of the Conan cache. But it is not good that these binaries will be copied everytime a normal build of this package happens in the cache.
• self.requires("openssl/3.0.14@user/channel#ffb3f96cda0658e55369c4ece26c3292" hardcoding recipe-revisions in recipes shouldn't be necessary in most cases. For reproducibility of the whole dependency set it would be better to use lockfiles.

Regarding the FIPS issue, I am not sure what you are trying to achieve. I understood that it can be implemented through a dependency of openssl to itself in previous versions, in visible=False way to re-package the necessary artifacts. But I understood that this is done in the openssl recipe directly, no need for consumers to have to define, override, or anything like that. Maybe @jcar87 can provide more insights about this.

[Nekto89]

• copy(self, "*.dll", dep.cpp_info.bindirs[0], r"C:/dev/bld/Binaries/Release") this seems an anti-pattern. Conan got the deploy() method to control what/when to copy things in folders outside of the Conan cache. But it is not good that these binaries will be copied everytime a normal build of this package happens in the cache.

that's the whole idea. Conan is called during CMake generation and all the dlls are copied to folder where all the .exe/.dll will be built.

self.requires("openssl/3.0.14@user/channel#ffb3f96cda0658e55369c4ece26c3292" hardcoding recipe-revisions in recipes shouldn't be necessary in most cases. For reproducibility of the whole dependency set it would be better to use lockfiles.

conan is called multiple times with very dynamic list of packages, lockfiles would be very inconvenient to use.

But I understood that this is done in the openssl recipe directly, no need for consumers to have to define, override, or anything like that.

Yes, openssl recipe has dependency on "openssl/3.0.9". But I would like to override that dependency with my own with specific user/channel/revision to fully stabilize it and be able to build it in future if something would change. I don't want to hardcode user/channel/revision in openssl recipe without need if it's possible to make external override.

[memsharded]

that's the whole idea. Conan is called during CMake generation and all the dlls are copied to folder where all the .exe/.dll will be built.

This is not correct. Conan regular builds and generate() methods should put things in the generator_folder, but it shouldn't be copying files outside of the Conan cache in the system. This is not expected in Conan 2, this is why the deploy() method has specific triggers.

It might be ok to copy some dlls inside the current self.build_folder of the current package, but copying files in a system location is considered a deploy operation and should only be done by deploy() method or --deployer deployers. And regular package builds should be fully self-contained, they shouldn't rely on some files copied to an external folder outside of its own cache build folder.

Yes, openssl recipe has dependency on "openssl/3.0.9". But I would like to override that dependency with my own with specific user/channel/revision to fully stabilize it and be able to build it in future if something would change. I don't want to hardcode user/channel/revision in openssl recipe without need if it's possible to make external override.

The recommendation for doing this is using your own fork of conan-center-index, and using packages without user/channel (see https://docs.conan.io/2/devops/conancenter/hosting_binaries.html), not using replace_requires. The use case for replace_requires are listed in https://docs.conan.io/2/reference/config_files/profiles.html#replace-requires (The usage of this feature is intended for temporarily solving conflicts or replacing a specific dependency by a system one in some cross-build scenarios.)

I don't think [replace_requires] is a good mechanism for what you are intended to achieve, if there are requires like openssl/[>=1.1 <4] they can be effectively replaced by your replacement, as it is within the range. You might try to force an identical match openssl/[>=1.1 <4]: openssl/[>=1.1 <4] but that sounds more like a hack than a solution.