search

conan-io / conan-center-index

Recipes for the ConanCenter repository
https://conan.io/center
MIT License
960 stars 1.77k forks source link

protobuf recent update broke protoc integration #21737

Open pgeler opened 11 months ago

pgeler commented 11 months ago

Description

new implementation as https://github.com/conan-io/conan-center-index/pull/20089 of searching protoc actually finds system protoc instead the one build by conan, as a result usage of the protoc leads to error: incompatible with your Protocol Buffer headers. Please #error incompatible with your Protocol Buffer headers. .... linux ubuntu 22.04 with default protoc installed from apt-get

Package and Environment Details

Conan profile

default

Steps to reproduce

cmake build with dependencies

Logs

Click to expand log ``` Put your log output here ```
SpaceIm commented 11 months ago

You have to add protobuf to your build requirements.

pgeler commented 11 months ago

You have to add protobuf to your build requirements.

it is already there

SpaceIm commented 11 months ago

So it should work, please share your conan version, compiler etc, conanfile, CMakeLists and commands invoked (properly fill the issue template in short).

There might be an edge case when user relies on legacy cmake generator, since bin dir of protobuf package is appended to PATH instead of prepended.

Sfletche132 commented 11 months ago

Hi I'm seeing the same thing with my build.

import os
from conan import ConanFile

class MyApplication(ConanFile):
    settings = "os", "compiler", "build_type", "arch"

    name = "my_app"
    version = "1.0.0.0"

    # comma-separated list of requirements
    generators = "CMakeDeps"

    keep_imports = True

    default_options = {"grpc/*:cpp_plugin": True,
                       "grpc/*:shared": False,
                       "protobuf/*:shared": False,
                       "boost/*:shared": False,
                       "fakeit/*:integration": "catch"}

    def requirements(self):
        self.requires("boost/1.83.0")
        self.requires("grpc/1.54.3")
        self.requires("openssl/3.1.3")
        self.requires("protobuf/3.21.12")
        self.requires("sqlite3/3.41.1")
        self.requires("spdlog/1.12.0")
        self.requires("nlohmann_json/3.11.2")
        self.requires("eigen/3.4.0")
        self.requires("fmt/10.1.1")
        self.requires("cli11/2.3.2")
        self.requires("zlib/1.3", override=True)
        self.requires("asio/1.28.2")
        # Install libusb on windows, get it from the toolchain otherwise (handled inside device manager)
        if self.settings.arch == "x86_64":
            self.requires("libusb/1.0.25")
        if self.settings.os == "Windows":
            self.requires("sdl/2.0.16")
        # Testing libs
        self.requires("fakeit/2.4.0")
        self.requires("catch2/3.4.0")
        self.requires("fake_clock/1.01")

    def build_requirements(self):
        self.tool_requires("protobuf/3.21.12")
        self.tool_requires("grpc/1.54.3")

    def build(self):
        pass
        # self.output.warn("CC=>{}".format(os.getenv("CC")))

    def imports(self):
        self.copy("license*", dst="licenses", folder=True, ignore_case=True)

    def layout(self):
        # We make the assumption that if the compiler is msvc the
        # CMake generator is multi-config
        multi = True if self.settings.get_safe("compiler") == "msvc" else False
        if multi:
            self.folders.generators = os.path.join("build", "generators")
        else:
            self.folders.generators = os.path.join("build", str(self.settings.build_type), "generators")

I'm using Ninja and invoking protobuf_generate inside my cmakelists.txt

SpaceIm commented 11 months ago

Where is CMakeToolchain?

Sfletche132 commented 11 months ago

Where is CMakeToolchain?

Ah yes, I'll try with the toolchain, thanks

pgeler commented 10 months ago

this is the conan file 

from conan import ConanFile

my_packages = {
    "asio/1.28.2": {},
    "backward-cpp/1.6": {},
    "benchmark/1.8.2": {},
    "docopt.cpp/0.6.3": {},
    "expected-lite/0.6.3": {},
    "grpc/1.54.3": {},
    "gtest/1.14.0": {},
    "openssl/[>=3.0 <3.1]": {},
    "protobuf/3.21.12": {},
    "range-v3/0.12.0": {},
    "rxcpp/4.1.1": {},
    "spdlog/1.12.0": {},
    "zlib/[>=1.2.13 <2.0]": {},
}

class ConanPkgs(ConanFile):
    settings = "os", "compiler", "build_type", "arch"
    generators = "CMakeToolchain", "CMakeDeps"
    requires = list(my_packages.keys())
    default_options = dict(kv for d in my_packages.values() for kv in d.items())

    def configure(self):
        if self.settings.os != "Windows":
            self.options["backward-cpp"].stack_details = "backtrace_symbol"
SpaceIm commented 10 months ago

protobuf is not in build requirements

pgeler commented 10 months ago

sorry what about? "protobuf/3.21.12": {},

protobuf is not in build requirements

pgeler commented 10 months ago

updating conanfile with build requirements did not solve the problem

    def build_requirements(self):
        self.tool_requires("protobuf/3.21.12")
        self.tool_requires("grpc/1.54.3")
Sfletche132 commented 10 months ago

I'm using conan through the https://github.com/conan-io/cmake-conan repo. It says the toolchain isn't supported with that method. Is there a workaround to use the above repo with the current protobuf changes?

pgeler commented 10 months ago

Im looking at revisions, so hopefully can use those... while package is fixed. will give more details in an hour or so

pgeler commented 10 months ago

ok I was able to get back to previous revision, just specify it as package reference, also may need to explicitly enable revisions conan config set general.revisions_enabled=True

"protobuf/3.21.12@#ed2404ae366627005beb2173eab68174"
pgeler commented 10 months ago

@SpaceIm any thoughts?

SpaceIm commented 10 months ago

I don't know what you are doing. You have not shared all elements I asked.

pgeler commented 10 months ago

Does it matter? The problem is that receipt is using incorrect version of protoc and not the one from the package, previously that was not the case, so this is the line which causing the problem https://github.com/conan-io/conan-center-index/blob/5ff3625b6cf3940055fa98e7c49e5648ab1727e2/recipes/protobuf/all/conanfile.py#L136, so it would be great if you check it one more time from perspective of this issue. I can submit the fix, but it can break your intention, which I don't want to, as I have no visibility to original intention.

ps. I don't think it is a productive conversation to point that everybody should just switch over to cmake toolchains because it is working.

leoparente commented 10 months ago

I'm having the same problem as @pgeler and what he mentioned makes sense.

I have protoc 3.12 installed in my system, but for my project I use conan protobuf 3.21. The PR https://github.com/conan-io/conan-center-index/pull/20089 simply broke my build.. I use legacy cmake with conanfile.txt. I will do a workaround for now, but that is it.

tgockel commented 10 months ago

I'm not that familiar with Conan, but in my case protobuf comes as a transitive dependency through grpc, but I can't figure out how to force a protobuf version to one that works without causing a conflict. For example:

conan_cmake_configure(
  REQUIRES
    "grpc/1.50.1@#01c169a35963fcf8fca63a6f67a7e592"
    "protobuf/3.21.9@#64ce20e1d9ea24f3d6c504015d5f6fa8"
  GENERATORS
    CMakeDeps
    CMakeToolchain
)

This is a bit odd, since grpc/1.50.1@#01c169a35963fcf8fca63a6f67a7e592 was published 2023-06-11, but protobuf/3.21.9@#64ce20e1d9ea24f3d6c504015d5f6fa8 was published 2023-11-21, so clearly I'm missing something.

Trying to specify any other recipe revision leads to:

Graph error
    Version conflict: Conflict between protobuf/3.21.9 and protobuf/3.21.9 in the graph.

Conan version 2.0.14. CMake command is just cmake .. -- you don't have to do anything special to show the problem, Protobuf_PROTOC_EXECUTABLE will point to /usr/bin/protoc instead of the Conan location if protobuf-compiler is installed.

tgockel commented 10 months ago

For future readers using cmake-conan: There is no syntax in conanfile.txt that allows you to set a specific version of protobuf and not have it conflict with the Conan-chosen transitive dependency that comes through the grpc package. You must move to conanfile.py.

From the original CMake:

conan_cmake_configure(
  REQUIRES
    grpc/1.50.1
  GENERATORS
    CMakeDeps
    CMakeToolchain
)

Make the conanfile.py in your root:

from conan import ConanFile

class Package(ConanFile):
    settings = "os", "compiler", "build_type", "arch"
    generators = 'CMakeDeps', 'CMakeToolchain'

    def requirements(self):
        self.requires("grpc/1.50.1")
        # grab recipe version from `conan list -r conancenter 'protobuf/${protobuf_version}#*'`
        self.requires("protobuf/3.21.12@#8210c0b1bb46b08ff2814614ec091a9c", override=True)

Then, you can replace the call to conan_cmake_configure with a configure_file:

configure_file(conanfile.py "${CMAKE_CURRENT_BINARY_DIR}/conanfile.py" COPYONLY)
file(REMOVE "${CMAKE_CURRENT_BINARY_DIR}/conanfile.txt")

The file(REMOVE ...) makes sure the old conanfile.txt is cleaned up (you can't have both conanfile.txt and conanfile.py). The use of configure_file will make sure that CMake will properly re-run if conanfile.py changes (this won't happen if you conan_cmake_install(PATH_OR_REFERENCE "${CMAKE_CURRENT_LIST_DIR}" ...)).

uilianries commented 10 months ago

@pgeler Hello! Thank you for reporting! When opening the issue, please, fill the form with all asked information. Otherwise, it will be really hard to reproduce your scenario and confirm your error, plus, work in a hotfix. Could you please update your issue description with all information like: your real profile (conan profile show -pr:h default), the environment (if need to update), and really important: an entire build log with your error.

ilya-lavrenov commented 9 months ago

With updated protobuf recipe OpenVINO cannot be built on WIndows anymore https://github.com/conan-io/conan-center-index/pull/22244 even for old releases.

SpaceIm commented 9 months ago

see https://github.com/conan-io/conan-center-index/pull/22542#issuecomment-1910760666 and https://github.com/conan-io/conan-center-index/pull/22244#issuecomment-1910770387

dpad commented 9 months ago

I had the same issue and confirmed that adding protobuf to the build_requirements is the correct fix. It's not enough to have it only in requirements, it needs to be in build_requirements as well because the protoc compiler is used for the build process. I am not sure if this is the exact same problem as what is occurring to some of the users above, but the code below fixed it for me.

The following (based on @SpaceIm 's comment in https://github.com/conan-io/conan-center-index/pull/22244#issuecomment-1910770387) fixed this issue for me (for both Conan 1 and 2):

@property 
def _is_legacy_one_profile(self): 
    # Returns True if user is running a Conan 1 profile
    return not hasattr(self, "settings_build") 

def build_requirements(self):
    # Include protobuf as a build requirement so that Conan-provided
    # `protoc` is found instead of system-installed one).
    if not self._is_legacy_one_profile: 
        self.tool_requires("protobuf/<host_version>")