[bug] msvc compatible package not working with package_id()

[bruchar1]

Describe the bug

conan 1.65. Windows.

If I set compiler to msvc and I try to use require boost, it doesn't find the compatible package (with the Visual Studio compiler), because sha of the compatible package contains keys that are filtered out in the original package by the package_id() function.

I think that in graph_binaries.py:_compute_package_id(), the call to conanfile.package_id() should be done before generating the compatible packages for msvc and apple_clang.

How to reproduce it

conanfile.txt:

[requires]
boost/1.86.0

profile:

[settings]
arch=x86_64
os=Windows
compiler=msvc
compiler.version=192
compiler.runtime=dynamic
compiler.runtime_type=Release
compiler.cppstd=14

> conan . --profile=profile -sbuild_type=Release

The compatible package should be installed, but is not found...

[memsharded]

Hi @bruchar1

Thanks for your feedback.

I am not sure what compatibility you refer to. Conan 1.X doesn't define any default compatibility for the msvc compiler. This is only a Conan 2.X feature (the compatibility happens for different cppstd values and msvc 194->193 compiler.version)

If you are talking about binaries built in ConanCenter, note that in Conan 1.X, ConanCenter build binaries with the legacy compiler="Visual Studio" compiler, not the msvc compiler, and although some attempt was made to have compatibility, it is limited and doesn't happen in all situations, so the recommendation would be to use compiler="Visual Studio", please try that and let us know.

I'd like to clarify also that in many cases for production systems, the recommendation is to build your own binaries for the configurations you need and host them in your own servers. The guides in https://docs.conan.io/2/devops/using_conancenter.html was written in the Conan 2 docs, but it is also applicable for Conan 1.

[bruchar1]

We are in the process to migrate to conan 2. Our plan is to convert our dependencies to make them compatible with conan 2. Therefore, they will need to use the "msvc" compiler. I was trying to switch to "msvc" for our main project, to allow the conversion of our dependencies, while relying on the binary compatibility mode to use the existing "Visual Studio" binaries when possible. But I realized that for some package, like boost or opencv, the binary hash computed to find the compatible package is wrong, because it is computed before the package_id() function is called. Therefore, it computes the hash on a different set of options.

We already have our own Artifactory server, mirroring conan center, and providing binaries for our own packages or for different sets of options. However, being able to rely on the compatibility mode for Visual Studio would simplify the process of migrating to conan 2; otherwise we will need to switch all the dependencies at the same time, preventing us to migrate them one by one.

[memsharded]

Therefore, they will need to use the "msvc" compiler. I was trying to switch to "msvc" for our main project, to allow the conversion of our dependencies, while relying on the binary compatibility mode to use the existing "Visual Studio" binaries when possible.

This shouldn't be necessary. It is important to know that the binaries of Conan 1 and Conan 2 are mutually exclusive. Even if using the exact same input profile, the package_id for Conan 1 and Conan 2 will be different. Even more, conan search in Conan 1 will only list package binaries built with Conan 1 and conan list in conan 2 will only list package binaries built with Conan 2.

In practice, most users stayed in compiler=Visual Studio in Conan1, and used compiler=msvc in Conan 2. It is also quite expected that the profiles are not identical either in Conan 1 and Conan 2. For example, Conan 1 didn't use compiler.cppstd in the default profiles, while in Conan 2 it is almost mandatory. There are a few other things that changed, like all profiles are always jinja-rendered, irrespective of the extension.

However, being able to rely on the compatibility mode for Visual Studio would simplify the process of migrating to conan 2; otherwise we will need to switch all the dependencies at the same time, preventing us to migrate them one by one.

Even if the package binaries are different for Conan 1 and 2, it is possible to migrate them one at a time. The recommendation is to use the same recipes, and it is possible to use also the same server repositories, exactly because the package_id are mutually exclusive. Setting up a parallel Conan 2 pipeline in CI can start creating the packages for Conan 2 while not disrupting operations with Conan 1.

Please let me know if this helps.

[bruchar1]

Ok. If I understand correctly, I can use the same recipe for both conan 1 and conan 2, but I should use different profiles: one with Visual Studio compiler for conan 1, and one with msvc compiler for conan 2. I guess I can do that.

What confused me is the fact that msvc compiler is available in conan 1. That's why I taught it was recommended to use it.

That being said, I still think there is a bug in the way the checksum for compatible binaries is computed. But I agree there is a workaround by simply using Visual Studio compiler instead of relying on the compatibility mode for msvc.

[memsharded]

Ok. If I understand correctly, I can use the same recipe for both conan 1 and conan 2, but I should use different profiles: one with Visual Studio compiler for conan 1, and one with msvc compiler for conan 2. I guess I can do that.

Yes, exactly. But not only msvc, but also compiler.cppstd defined is very recommended in Conan 2, while if you try to apply it for Conan 1, it might break your pipelines (missing binaries, etc)

What confused me is the fact that msvc compiler is available in conan 1. That's why I taught it was recommended to use it.

Yes, we tried to implement it in Conan 1, but the limitations of Conan 1 made it impossible to make it as transparent and smooth as we wanted.

That being said, I still think there is a bug in the way the checksum for compatible binaries is computed. But I agree there is a workaround by simply using Visual Studio compiler instead of relying on the compatibility mode for msvc.

Agree this is very likely the best approach. Even for known bugs, if they aren't quite straightforward to fix and with zero risk to break other users, they are no longer being addressed in Conan 1, as Conan 2 is already like 20 months old.