conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.28k stars 981 forks source link

[question] Managing multiple C++ packages within the same git repository #12521

Closed KingKiller100 closed 2 years ago

KingKiller100 commented 2 years ago

Could you please advise? If any more information is need (such as seeing the contents of the conanfile.py of each package) please let me know.

Directory Structure

  .
  |-- PackageA
      |- conanfile.py
  |-- PackageB
      |- conanfile.py

Logs

$ conan install -pr release_141 .
DEBUG :conan_api.py   [176]: INIT: Using config 'd:\.conan\.conan\conan.conf' [2022-11-12 20:59:51,409]
DEBUG :tracer.py      [157]: CONAN_API: install(path=.,name=None,version=None,user=None,channel=None,settings=None,options=None,env=None,profile_names=['release_141'],conf=None,profile_build=ProfileData(profiles=None, settings=None, options=None, env=None, conf=None),remote_name=None,verify=None,manifests=None,manifests_interactive=None,build=None,update=False,generators=None,no_imports=False,install_folder=None,output_folder=None,lockfile=None,lockfile_out=None,require_overrides=None) [2022-11-12 20:59:51,411]
DEBUG :profile_loader.py[120]: PROFILE LOAD: d:\.conan\.conan\profiles\default [2022-11-12 20:59:51,413]
DEBUG :profile_loader.py[120]: PROFILE LOAD: d:\.conan\.conan\profiles\release_141 [2022-11-12 20:59:51,414]
DEBUG :profile_loader.py[120]: PROFILE LOAD: d:\.conan\.conan\profiles\default [2022-11-12 20:59:51,452]
Configuration:
[settings]
arch=x86
arch_build=x86_64       
build_type=Release      
compiler=Visual Studio  
compiler.runtime=MT     
compiler.toolset=v141_xp
compiler.version=15     
os=Windows
os_build=Windows
[options]
[build_requires]
[env]

DEBUG :graph_builder.py[462]: GRAPH: new_node: PackageA/none@local/edit [2022-11-12 20:59:54,693]
DEBUG :graph_builder.py[462]: GRAPH: new_node: icu/58.3@inseinc/stable [2022-11-12 20:59:54,702]
DEBUG :graph_builder.py[462]: GRAPH: new_node: directx/9.0.0-2@inseinc/stable [2022-11-12 20:59:54,708]
DEBUG :graph_builder.py[462]: GRAPH: new_node: boost/1.71.0@conan/stable [2022-11-12 20:59:54,721]
DEBUG :graph_builder.py[462]: GRAPH: new_node: zlib/1.2.11@conan/stable [2022-11-12 20:59:54,728]
DEBUG :graph_builder.py[462]: GRAPH: new_node: bzip2/1.0.8@conan/stable [2022-11-12 20:59:54,736]
DEBUG :graph_builder.py[462]: GRAPH: new_node: cryptopp/5.6.5@inseinc/stable [2022-11-12 20:59:54,742]
DEBUG :graph_builder.py[462]: GRAPH: new_node: libb64/1.2.1-1@inseinc/stable [2022-11-12 20:59:54,748]
DEBUG :graph_builder.py[462]: GRAPH: new_node: mpir/2.7.0-2@inseinc/stable [2022-11-12 20:59:54,756]
DEBUG :graph_builder.py[462]: GRAPH: new_node: tinyxml/2.6.2@inseinc/stable [2022-11-12 20:59:54,762]
DEBUG :graph_builder.py[462]: GRAPH: new_node: xvid/1.3.7@inseinc/stable [2022-11-12 20:59:54,769]
DEBUG :graph_builder.py[69]: GRAPH: Time to load deps 0.07996344566345215 [2022-11-12 20:59:54,770]
DEBUG :graph_builder.py[462]: GRAPH: new_node: crc-calculator/1.0.1@inseinc/stable [2022-11-12 20:59:54,800]
conanfile.py (pal-core/12.2.25-alpha.9+git.5d89a8b): Installing package
Requirements
    boost/1.71.0@conan/stable from 'inseinc' - Cache
    bzip2/1.0.8@conan/stable from 'inseinc' - Cache
    cryptopp/5.6.5@inseinc/stable from 'inseinc' - Cache
    directx/9.0.0-2@inseinc/stable from 'inseinc' - Cache
    icu/58.3@inseinc/stable from 'inseinc' - Cache
    libb64/1.2.1-1@inseinc/stable from 'inseinc' - Cache
    mpir/2.7.0-2@inseinc/stable from 'inseinc' - Cache
    tinyxml/2.6.2@inseinc/stable from 'inseinc' - Cache
    xvid/1.3.7@inseinc/stable from 'inseinc' - Cache
    zlib/1.2.11@conan/stable from 'inseinc' - Cache
    PackageA/none@local/edit from user folder - Editable
Python requires
    version-calculator/1.5.0@inseinc/stable
Packages
    boost/1.71.0@conan/stable:44aa2302561ce659d0e223e4bd316cf9d6030751 - Cache
    bzip2/1.0.8@conan/stable:802bee0999b6305c4060a347404f0b86312d67d0 - Cache
    cryptopp/5.6.5@inseinc/stable:d3fb7beef1deae181d55400b6e03fa5d5d5cf8cc - Cache
    directx/9.0.0-2@inseinc/stable:63c5d5dc6eb958dab897e143053e17f583d55879 - Cache
    icu/58.3@inseinc/stable:4803f0b3235c36c19a969902b9cf47ed10cdec17 - Cache
    libb64/1.2.1-1@inseinc/stable:d3fb7beef1deae181d55400b6e03fa5d5d5cf8cc - Cache
    mpir/2.7.0-2@inseinc/stable:d3fb7beef1deae181d55400b6e03fa5d5d5cf8cc - Cache
    tinyxml/2.6.2@inseinc/stable:d3fb7beef1deae181d55400b6e03fa5d5d5cf8cc - Cache
    xvid/1.3.7@inseinc/stable:d3fb7beef1deae181d55400b6e03fa5d5d5cf8cc - Cache
    zlib/1.2.11@conan/stable:d3fb7beef1deae181d55400b6e03fa5d5d5cf8cc - Cache
    PackageA/none@local/edit:cb989ff2423724a8585c9b30595c6898909e5b6a - Editable
Build requirements
    crc-calculator/1.0.1@inseinc/stable from 'inseinc' - Cache
Build requirements packages
    crc-calculator/1.0.1@inseinc/stable:11e6a84a7894f41df553e7c92534c3bf26896802 - Cache

Cross-build from 'Windows:x86_64' to 'Windows:x86'
Installing (downloading, building) binaries...
bzip2/1.0.8@conan/stable: Already installed!
crc-calculator/1.0.1@inseinc/stable: Already installed!
cryptopp/5.6.5@inseinc/stable: Already installed!
directx/9.0.0-2@inseinc/stable: Already installed!
icu/58.3@inseinc/stable: Already installed!
libb64/1.2.1-1@inseinc/stable: Already installed!
mpir/2.7.0-2@inseinc/stable: Already installed!
tinyxml/2.6.2@inseinc/stable: Already installed!
xvid/1.3.7@inseinc/stable: Already installed!
zlib/1.2.11@conan/stable: Already installed!
boost/1.71.0@conan/stable: Already installed!
boost/1.71.0@conan/stable: LIBRARIES: ['libboost_wave', 'libboost_container', 'libboost_contract', 'libboost_exception', 'libboost_graph', 'libboost_iostreams', 'libboost_locale', 'libboost_log', 'libboost_program_options', 'libboost_random', 'libboost_regex', 'libboost_serialization', 'libboost_wserialization', 'libboost_coroutine', 'libboost_fiber', 'libboost_context', 'libboost_timer', 'libboost_thread', 'libboost_chrono', 'libboost_date_time', 'libboost_atomic', 'libboost_filesystem', 'libboost_system', 'libboost_type_erasure', 'libboost_log_setup', 'libboost_math_c99', 'libboost_math_c99f', 'libboost_math_c99l', 'libboost_math_tr1', 'libboost_math_tr1f', 'libboost_math_tr1l', 'libboost_stacktrace_noop', 'libboost_stacktrace_windbg', 'libboost_stacktrace_windbg_cached', 'libboost_unit_test_framework']
boost/1.71.0@conan/stable: Package folder: d:\.conan\a8b182\1
boost/1.71.0@conan/stable: Disabled magic autolinking (smart and magic decisions)
PackageA/none@local/edit: Applying build-requirement: crc-calculator/1.0.1@inseinc/stable
PackageA/none@local/edit: Rewriting files of editable package 'PackageA' at 'D:\Git\Frameworks\Library\build/PackageA/generated'
PackageA/none@local/edit: Generator txt created conanbuildinfo.txt
PackageA/none@local/edit: Calling generate()
PackageA/none@local/edit: WARN: Using the new toolchains and generators without specifying a build profile (e.g: -pr:b=default) is discouraged and might cause failures and unexpected behavior
PackageA/none@local/edit: WARN: Using the new toolchains and generators without specifying a build profile (e.g: -pr:b=default) is discouraged and might cause failures and unexpected behavior
PackageA/none@local/edit: MSBuildToolchain created conantoolchain_release_v141_win32.props
PackageA/none@local/edit: MSBuildToolchain writing conantoolchain.props
INFO  :runners.py     [87]: Calling command: "C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe" -format json -legacy -nologo > "C:\Users\KAREL~1.DEH\AppData\Local\Temp\tmp_zu9yi1c\output" [2022-11-12 20:59:55,025]
INFO  :runners.py     [90]: Return code: 0 [2022-11-12 20:59:55,157]
INFO  :runners.py     [98]: Output: in file:[
                              {
                                "instanceId": "15c7e559",
                                "installDate": "2021-08-03T15:01:39Z",
                                "installationName": "VisualStudio/15.9.51+33027.88",
                                "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional",
                                "installationVersion": "15.9.33027.88",
                                "productId": "Microsoft.VisualStudio.Product.Professional",
                                "productPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\Common7\\IDE\\devenv.exe",
                                "state": 4294967295,
                                "isComplete": true,
                                "isLaunchable": true,
                                "isPrerelease": false,
                                "isRebootRequired": false,
                                "displayName": "Visual Studio Professional 2017",
                                "description": "Professional developer tools and services for small teams",
                                "channelId": "VisualStudio.15.Release",
                                "channelPath": "C:\\Users\\xyz\\AppData\\Local\\Microsoft\\VisualStudio\\Packages\\_Channels\\4CB340F5\\install_catalog.json",
                                "channelUri": "https://aka.ms/vs/15/release/channel",
                                "enginePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\resources\\app\\ServiceHub\\Services\\Microsoft.VisualStudio.Setup.Service",
                                "installChannelUri": "\\\\WSUSBUR\\vs2k17$\\ChannelManifest.json",
                                "layoutPath": "\\\\WSUSBUR\\vs2k17$",
                                "releaseNotes": "https://go.microsoft.com/fwlink/?LinkId=660692#15.9.51",
                                "thirdPartyNotices": "https://go.microsoft.com/fwlink/?LinkId=660708",
                                "updateDate": "2022-11-12T18:55:44.7601719Z",
                                "catalog": {
                                  "buildBranch": "d15.9",
                                  "buildVersion": "15.9.33027.88",
                                  "id": "VisualStudio/15.9.51+33027.88",
                                  "localBuild": "build-lab",
                                  "manifestName": "VisualStudio",
                                  "manifestType": "installer",
                                  "productDisplayVersion": "15.9.51",
                                  "productLine": "Dev15",
                                  "productLineVersion": "2017",
                                  "productMilestone": "RTW",
                                  "productMilestoneIsPreRelease": "False",
                                  "productName": "Visual Studio",
                                  "productPatchVersion": "51",
                                  "productPreReleaseMilestoneSuffix": "1.0",
                                  "productRelease": "RTW",
                                  "productSemanticVersion": "15.9.51+33027.88",
                                  "requiredEngineVersion": "1.18.1063.29791"
                                },
                                "properties": {
                                  "campaignId": "",
                                  "channelManifestId": "VisualStudio.15.Release/15.9.51+33027.88",
                                  "nickname": "",
                                  "setupEngineFilePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vs_installershell.exe"
                                }
                              },
                              {
                                "instanceId": "VisualStudio.14.0",
                                "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\",
                                "installationVersion": "14.0"
                              },
                              {
                                "instanceId": "VisualStudio.12.0",
                                "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\",
                                "installationVersion": "12.0"
                              },
                              {
                                "instanceId": "VisualStudio.10.0",
                                "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\",
                                "installationVersion": "10.0"
                              }
                            ]

                            stdout: None
                            stderr:b'' [2022-11-12 20:59:55,171]
INFO  :runners.py     [87]: Calling command: "C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe" -format json -products * -nologo > "C:\Users\KAREL~1.DEH\AppData\Local\Temp\tmplyghz5fl\output" [2022-11-12 20:59:55,183]
INFO  :runners.py     [90]: Return code: 0 [2022-11-12 20:59:55,281]
INFO  :runners.py     [98]: Output: in file:[
                              {
                                "instanceId": "15c7e559",
                                "installDate": "2021-08-03T15:01:39Z",
                                "installationName": "VisualStudio/15.9.51+33027.88",
                                "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional",
                                "installationVersion": "15.9.33027.88",
                                "productId": "Microsoft.VisualStudio.Product.Professional",
                                "productPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\Common7\\IDE\\devenv.exe",
                                "state": 4294967295,
                                "isComplete": true,
                                "isLaunchable": true,
                                "isPrerelease": false,
                                "isRebootRequired": false,
                                "displayName": "Visual Studio Professional 2017",
                                "description": "Professional developer tools and services for small teams",
                                "channelId": "VisualStudio.15.Release",
                                "channelPath": "C:\\Users\\xyz\\AppData\\Local\\Microsoft\\VisualStudio\\Packages\\_Channels\\4CB340F5\\install_catalog.json",
                                "channelUri": "https://aka.ms/vs/15/release/channel",
                                "enginePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\resources\\app\\ServiceHub\\Services\\Microsoft.VisualStudio.Setup.Service",
                                "installChannelUri": "\\\\WSUSBUR\\vs2k17$\\ChannelManifest.json",
                                "layoutPath": "\\\\WSUSBUR\\vs2k17$",
                                "releaseNotes": "https://go.microsoft.com/fwlink/?LinkId=660692#15.9.51",
                                "thirdPartyNotices": "https://go.microsoft.com/fwlink/?LinkId=660708",
                                "updateDate": "2022-11-12T18:55:44.7601719Z",
                                "catalog": {
                                  "buildBranch": "d15.9",
                                  "buildVersion": "15.9.33027.88",
                                  "id": "VisualStudio/15.9.51+33027.88",
                                  "localBuild": "build-lab",
                                  "manifestName": "VisualStudio",
                                  "manifestType": "installer",
                                  "productDisplayVersion": "15.9.51",
                                  "productLine": "Dev15",
                                  "productLineVersion": "2017",
                                  "productMilestone": "RTW",
                                  "productMilestoneIsPreRelease": "False",
                                  "productName": "Visual Studio",
                                  "productPatchVersion": "51",
                                  "productPreReleaseMilestoneSuffix": "1.0",
                                  "productRelease": "RTW",
                                  "productSemanticVersion": "15.9.51+33027.88",
                                  "requiredEngineVersion": "1.18.1063.29791"
                                },
                                "properties": {
                                  "campaignId": "",
                                  "channelManifestId": "VisualStudio.15.Release/15.9.51+33027.88",
                                  "nickname": "",
                                  "setupEngineFilePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vs_installershell.exe"
                                }
                              }
                            ]

                            stdout: None
                            stderr:b'' [2022-11-12 20:59:55,293]
PackageA/none@local/edit: Aggregating env generators
PackageA/none@local/edit: Generated toolchain
PackageA/none@local/edit: Generated conan.lock
PackageA/none@local/edit imports(): Copied 1 '.exe' file: CrcCalculator.exe
PackageA/none@local/edit: Copied 1 '.exe' file: CrcCalculator.exe
Traceback (most recent call last):
  File "conans\client\command.py", line 2216, in run
  File "conans\client\command.py", line 539, in install
  File "conans\client\conan_api.py", line 93, in wrapper
  File "conans\client\conan_api.py", line 625, in install
  File "conans\client\manager.py", line 127, in deps_install
  File "conans\client\generators\__init__.py", line 160, in write_generators
  File "conans\client\generators\__init__.py", line 233, in _receive_conf
AttributeError: 'list' object has no attribute 'direct_build'

ERROR: 'list' object has no attribute 'direct_build'
memsharded commented 2 years ago

Hi @KingKiller100

I suspect that there could be something in the package_info() of the PackageA that is affecting the propagation, probably something related to the conf. Yes, it would be great to see your conanfile.py, that could give some further ideas.

Also, just a quick suggestion for the future, regarding to package name PackageA: try to use package names in lowercase only, because ConanCenter has been forcing that for years, and Conan 2.0 will force lowercase packages (there will be a temporary workaround, but the recommendation is lowercase)

KingKiller100 commented 2 years ago

Good evening,

Thank you so much for answering back. Thank you for the information about package names, I will make sure to enforce all packages are lowercase in future to avoid any trouble.

Update

Whilst I was waiting for your response, I figured out the cause of the issue was a name clashing with something Conan uses and I was typing the object to be a list. the object was called "dependencies". This is why the error occurred.

I have gotten past that issue and now suffering a new one that looks similar but I'm struggling to find the source of the issue within the scripts. Now my issue is with the MSBuildDeps generator.

Could you aid me please? Is there a list of reserved names by conan that I can reference to prevent issues like this happening again?

Here are the contents of my scripts.

package-a

from conans import ConanFile, tools, MSBuild
from conan.tools.microsoft import MSBuildToolchain, MSBuildDeps
import os

required_conan_version = ">=1.42.0"

class Package_A(ConanFile):
    name = "package-a"
    description = "The library core utilities."
    license = "Copyright Inspired Entertainment Inc. 2022"
    settings = "os", "compiler", "build_type", "arch"
    short_paths = True
    keep_imports = True
    scm = {
        "type": "git",
        "url": "auto",
        "revision": "auto",
    }
    python_requires = "version-calculator/1.5.0@inseinc/stable"
    options = {
        "flamegraph_buffer_size_bits": "ANY",
        "flamegraph_verbosity": [ "verbose", "verbatim" ],
        "profiler": [ "none", "full", "drprofile", "drmemory", "drcallstack", "drloadtime" ],
    }
    default_options = {
        "flamegraph_buffer_size_bits": 0,
        "flamegraph_verbosity": "verbose",
        "profiler": "none",
    }
    profiler_flamegraph_verbosities = {
        "verbose": [ "SCOPE_ENABLE_FLAMEGRAPH" ],
        "verbatim": [ "SCOPE_ENABLE_FLAMEGRAPH", "FLAMEGRAPH_VERBATIM" ],
    }
    profiler_modes  = {
        "full": ["SCOPE_ENABLE_PROFILE", "SCOPE_ENABLE_CALLSTACK", "SCOPE_ENABLE_TRACKBACK", "SCOPE_ENABLE_MEMTRACKING","ENABLE_LOADMETERING"],
        "drprofile": ["SCOPE_ENABLE_PROFILE"],
        "drmemory": ["SCOPE_ENABLE_MEMTRACKING"],
        "drcallstack": ["SCOPE_ENABLE_CALLSTACK"],
        "drtrackback": ["SCOPE_ENABLE_TRACKBACK"],
        "drloadtime": ["ENABLE_LOADMETERING"],
    }
    subprojects = [ 
        f"Internationalization",
        f"Internationalization_Icu",
        f"RevCore",
        f"Win32_Helper",
    ]

    def set_version(self):
        self.version = "12.2.25"

    def configure(self):
        self.options["icu"].shared = False
        self.options["icu"].data_packaging = "static"

        # TODO: this is a hack required by the old msbuild helper
        # when we update, this should be moved to MSBuildDeps and MSBuildToolchain, and the new MSBuild
        if self.settings.compiler.toolset == "v141_xp":
            self.settings.build_type = f"{self.settings.build_type}_v141"
        else:
            raise Exception("unknown toolset")

    def layout(self):
        self.folders.root = "../.."
        self.folders.generators = "build/package_a/generated"

    def build_requirements(self):
        self.build_requires("crc-calculator/1.0.1@inseinc/stable")

    def requirements(self):
        self.requires("icu/58.3@inseinc/stable")
        self.requires("directx/9.0.0-2@inseinc/stable")
        self.requires("boost/1.71.0@conan/stable")
        self.requires("cryptopp/5.6.5@inseinc/stable")
        self.requires("libb64/1.2.1-1@inseinc/stable")
        self.requires("mpir/2.7.0-2@inseinc/stable")
        self.requires("tinyxml/2.6.2@inseinc/stable")
        self.requires("xvid/1.3.7@inseinc/stable")

    def imports(self):
        self.copy(pattern="CrcCalculator.exe", src="bin", dst="temp")

    def generate(self):
        MSBuildDeps(self).generate()

        tc = MSBuildToolchain(self)

        if int(self.options.flamegraph_buffer_size_bits) > 0:
            tc.preprocessor_definitions["FLAMEGRAPH_BUFFER_SIZE_BITS"] = self.options.flamegraph_buffer_size_bits
            definitions = dict.fromkeys(self.profiler_flamegraph_verbosities[str(self.options.flamegraph_verbosity)])
            tc.preprocessor_definitions.update(definitions)

        if str(self.options.profiler) in self.profiler_modes:
            preprocessors = dict.fromkeys(self.profiler_modes[str(self.options.profiler)])
            tc.preprocessor_definitions.update(preprocessors)

        tc.generate()

    def build(self):
        if tools.get_env("INSEINC_SIMULATE_BUILD") == "1":
            print("Skipping build: INSEINC_SIMULATE_BUILD is set.")
            return

        solution_file = "build/package_a/Raptor.RevCore.sln"
        msbuild = MSBuild(self)
        msbuild.build(
            solution_file,
            upgrade_project=False,
            toolset=self.settings.compiler.toolset,
            platforms={"x86": "Win32"},
            use_env=False,
            force_vcvars=True)

    def package(self):
        output_dir_with_build_type = f"temp/MsBuild/Output/{self.settings.build_type}"

        for project in self.subprojects:
            self.copy(
                pattern="*.h",
                dst=f"include/{project}",
                src=f"source/{project}",
                keep_path=True)
            self.copy(
                pattern="*.hpp",
                dst=f"include/{project}",
                src=f"source/{project}",
                keep_path=True)

        # also include the includes in the include directory...
        autoLinkList = ["_*"]
        autoLinkList.extend(self.subprojects),
        includeFiles: dict[str,list[str]] = {
            ".":["win32", "UndoWindowsBadness"], 
            "Autolink": autoLinkList, 
        }

        for dir, filenamePatterns in includeFiles.items():
            for pattern in filenamePatterns:
                self.copy(
                    pattern=f"{pattern}.h",
                    dst=f"include/{dir}",
                    src=f"include/{dir}",
                    keep_path=True)
                self.copy(
                    pattern=f"{pattern}.hpp",
                    dst=f"include/{dir}",
                    src=f"include/{dir}",
                    keep_path=True)

        self.copy(
            pattern=f"*.lib",
            dst="lib",
            src=output_dir_with_build_type,
            keep_path=False)

    def package_info(self):
        if not self.in_local_cache:
            self.cpp_info.includedirs = [ f"source", f"include" ]
            self.cpp_info.libdirs = [ f"temp/MsBuild/Output/{self.settings.build_type}" ]
            self.cpp_info.bindirs = [ f"temp/MsBuild/Output/{self.settings.build_type}" ]

        debug = f"{self.settings.build_type}".startswith("Debug")
        postfix = debug and "Debug" or "Release"

        if self.settings.compiler.toolset == "v141_xp":
            self.cpp_info.cxxflags = [
                "/std:c++17",
                "/Zc:__cplusplus",
                "/permissive"
            ]
        else:
            raise Exception("unknown toolset")

        self.cpp_info.defines = [ "_SILENCE_ALL_CXX17_DEPRECATION_WARNINGS" ]

        if int(self.options.flamegraph_buffer_size_bits) > 0:
            definitions = dict.fromkeys(self.profiler_flamegraph_verbosities[str(self.options.flamegraph_verbosity)])
            self.cpp_info.defines.extend(definitions)

        if str(self.options.profiler) in self.profiler_modes:
            self.cpp_info.defines.extend(self.profiler_modes[str(self.options.profiler)])

        for project in self.subprojects:
            self.cpp_info.libs.append(f"{project}_{postfix}")

    def package_id(self):
        # these cause full rebuilds when a dependency has changed even slightly, needed to prevent ABI but not API breakage
        self.info.requires.recipe_revision_mode()

package-b

from conans import ConanFile, tools, MSBuild
from conan.tools.files import update_conandata
import os

required_conan_version = ">=1.42.0"

def get_conanfile_version(dependency, conan_data, recipe_folder, update = False):
    # happens when it's editable
    if conan_data == None:
        return "none"
    return conan_data[f"{dependency}_version"]

class Package_B(ConanFile):
    name = "package-b"
    description = "Platform Abstraction Library core interfaces and primitives."
    license = "Copyright Inspired Entertainment Inc. 2022"
    settings = "os", "compiler", "build_type", "arch"
    short_paths = True
    keep_imports = True
    scm = {
        "type": "git",
        "url": "auto",
        "revision": "auto",
    }
    python_requires = "version-calculator/1.5.0@inseinc/stable"

    generators = "MSBuildToolchain","MSBuildDeps",

    _deps = [ "package-a" ]
    _subprojs = [ "package_b" ]
    default_user = "local"
    default_channel = "edit"

    options = {
        "flamegraph_buffer_size_bits": "ANY",
        "flamegraph_verbosity": [ "verbose", "verbatim" ],
        "profiler": [ "none", "full", "drprofile", "drmemory", "drcallstack", "drloadtime" ],
    }
    default_options = {
        "flamegraph_buffer_size_bits": 0,
        "flamegraph_verbosity": "verbose",
        "profiler": "none",
    }

    def configure(self):
        for dependency in self._deps:
            self.options[dependency].flamegraph_buffer_size_bits = self.options.flamegraph_buffer_size_bits
            self.options[dependency].flamegraph_verbosity = self.options.flamegraph_verbosity
            self.options[dependency].profiler = self.options.profiler

    def set_version(self):
        self.version = "12.2.25"

    def export(self):
        import subprocess
        conan_data = {}
        for dependency in self._deps:
            path = os.path.join(self.recipe_folder, f"../{dependency}/conanfile.py")
            command = f"conan inspect {path} --raw=version"
            conan_data["{dependency}_version"] = subprocess.run(
                command, stdout=subprocess.PIPE).stdout.decode('utf-8').strip()
        update_conandata(self, conan_data)

    def configure(self):
        # TODO: this is a hack required by the old msbuild helper
        # when we update, this should be moved to MSBuildDeps and MSBuildToolchain, and the new MSBuild
        if self.settings.compiler.toolset == "v141_xp":
            self.settings.build_type = f"{self.settings.build_type}_v141"
        else:
            raise Exception("unknown toolset")

    def layout(self):
        self.folders.root = "../.."
        self.folders.generators = "build/package_b/generated"

    def requirements(self):
        for dependency in self._deps:
            dep_version = get_conanfile_version(dependency, self.conan_data, self.recipe_folder)
            dep_request = (f"{dependency}/{dep_version}@local/edit")
            self.requires(dep_request)

    def package_id(self):
        # these cause full rebuilds when a dependency has changed even slightly, needed to prevent ABI but not API breakage
        self.info.requires.recipe_revision_mode()

    def build(self):
        if tools.get_env("INSEINC_SIMULATE_BUILD") == "1":
            print("Skipping build: INSEINC_SIMULATE_BUILD is set.")
            return

        solution_file = "build/package_b/package_b.sln"
        msbuild = MSBuild(self)
        msbuild.build(
            solution_file,
            upgrade_project=False,
            toolset=self.settings.compiler.toolset,
            platforms={"x86": "Win32"},
            use_env=False,
            force_vcvars=True)

    def package(self):
        output_dir_with_build_type = f"temp/MsBuild/Output/{self.settings.build_type}"
        for subproject in self._subprojs:
            self.copy(
                pattern="*.h",
                dst=f"include/{subproject}",
                src=f"source/{subproject}",
                keep_path=True)
            self.copy(
                pattern="*.hpp",
                dst=f"include/{subproject}",
                src=f"source/{subproject}",
                keep_path=True)
            self.copy(
                pattern=f"{subproject}_*.lib",
                dst="lib",
                src=output_dir_with_build_type,
                keep_path=False)

        # also include the includes in the include directory...
        includeFiles: dict[str,list[str]] = {
            "Autolink": self.subprojects, 
        }

        for dir, filenamePatterns in includeFiles.items():
            for pattern in filenamePatterns:
                self.copy(
                    pattern=f"{pattern}.h",
                    dst=f"include/{dir}",
                    src=f"include/{dir}",
                    keep_path=True)
                self.copy(
                    pattern=f"{pattern}.hpp",
                    dst=f"include/{dir}",
                    src=f"include/{dir}",
                    keep_path=True)

        self.copy(
            pattern=f"*.lib",
            dst="lib",
            src=output_dir_with_build_type,
            keep_path=False)

    def package_info(self):
        if not self.in_local_cache:
            self.cpp_info.includedirs = [ f"source", f"include" ]
            self.cpp_info.libdirs = [ f"temp/MsBuild/Output/{self.settings.build_type}" ]
            self.cpp_info.bindirs = [ f"temp/MsBuild/Output/{self.settings.build_type}" ]

        debug = f"{self.settings.build_type}".startswith("Debug")
        postfix = debug and "Debug" or "Release"

        self.cpp_info.libs = [
            f"package-b_{postfix}",
        ]

Logs:

$ conan install -pr release_141 .
DEBUG :conan_api.py   [176]: INIT: Using config 'd:\.conan\.conan\conan.conf' [2022-11-13 20:56:29,540]
DEBUG :tracer.py      [157]: CONAN_API: install(path=.,name=None,version=None,user=None,channel=None,settings=None,options=None,env=None,profile_names=['release_141'],conf=None,profile_build=ProfileData(profiles=None, settings=None, options=None, env=None, conf=None),remote_name=None,verify=None,manifests=None,manifests_interactive=None,build=None,update=False,generators=None,no_imports=False,install_folder=None,output_folder=None,lockfile=None,lockfile_out=None,require_overrides=None) [2022-11-13 20:56:29,542]      
DEBUG :profile_loader.py[120]: PROFILE LOAD: d:\.conan\.conan\profiles\release_141 [2022-11-13 20:56:29,545]
DEBUG :profile_loader.py[120]: PROFILE LOAD: d:\.conan\.conan\profiles\release_141 [2022-11-13 20:56:29,546]
DEBUG :profile_loader.py[120]: PROFILE LOAD: d:\.conan\.conan\profiles\release_141 [2022-11-13 20:56:29,588]
Configuration:
[settings]
arch=x86
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MT
compiler.toolset=v141_xp
compiler.version=15
os=Windows
os_build=Windows
[options]
[build_requires]
[env]

DEBUG :graph_builder.py[462]: GRAPH: new_node: package-a/none@local/edit [2022-11-13 20:56:32,449]
DEBUG :graph_builder.py[462]: GRAPH: new_node: icu/58.3@inseinc/stable [2022-11-13 20:56:32,458]
DEBUG :graph_builder.py[462]: GRAPH: new_node: directx/9.0.0-2@inseinc/stable [2022-11-13 20:56:32,464]
DEBUG :graph_builder.py[462]: GRAPH: new_node: boost/1.71.0@conan/stable [2022-11-13 20:56:32,475]
DEBUG :graph_builder.py[462]: GRAPH: new_node: zlib/1.2.11@conan/stable [2022-11-13 20:56:32,484]
DEBUG :graph_builder.py[462]: GRAPH: new_node: bzip2/1.0.8@conan/stable [2022-11-13 20:56:32,490]
DEBUG :graph_builder.py[462]: GRAPH: new_node: cryptopp/5.6.5@inseinc/stable [2022-11-13 20:56:32,497]
DEBUG :graph_builder.py[462]: GRAPH: new_node: libb64/1.2.1-1@inseinc/stable [2022-11-13 20:56:32,503]
DEBUG :graph_builder.py[462]: GRAPH: new_node: mpir/2.7.0-2@inseinc/stable [2022-11-13 20:56:32,510]
DEBUG :graph_builder.py[462]: GRAPH: new_node: tinyxml/2.6.2@inseinc/stable [2022-11-13 20:56:32,517]
DEBUG :graph_builder.py[462]: GRAPH: new_node: xvid/1.3.7@inseinc/stable [2022-11-13 20:56:32,523]
DEBUG :graph_builder.py[69]: GRAPH: Time to load deps 0.08885526657104492 [2022-11-13 20:56:32,524]
DEBUG :graph_builder.py[462]: GRAPH: new_node: crc-calculator/1.0.1@inseinc/stable [2022-11-13 20:56:32,556]
conanfile.py (package_b/12.2.25-alpha.9+git.48274b0): Installing package
Requirements
    boost/1.71.0@conan/stable from 'inseinc' - Cache
    bzip2/1.0.8@conan/stable from 'inseinc' - Cache
    cryptopp/5.6.5@inseinc/stable from 'inseinc' - Cache
    directx/9.0.0-2@inseinc/stable from 'inseinc' - Cache
    icu/58.3@inseinc/stable from 'inseinc' - Cache
    libb64/1.2.1-1@inseinc/stable from 'inseinc' - Cache
    mpir/2.7.0-2@inseinc/stable from 'inseinc' - Cache
    tinyxml/2.6.2@inseinc/stable from 'inseinc' - Cache
    xvid/1.3.7@inseinc/stable from 'inseinc' - Cache
    zlib/1.2.11@conan/stable from 'inseinc' - Cache
    package-a/none@local/edit from user folder - Editable
Python requires
    version-calculator/1.5.0@inseinc/stable
Packages
    boost/1.71.0@conan/stable:44aa2302561ce659d0e223e4bd316cf9d6030751 - Cache
    bzip2/1.0.8@conan/stable:802bee0999b6305c4060a347404f0b86312d67d0 - Cache
    cryptopp/5.6.5@inseinc/stable:d3fb7beef1deae181d55400b6e03fa5d5d5cf8cc - Cache
    directx/9.0.0-2@inseinc/stable:63c5d5dc6eb958dab897e143053e17f583d55879 - Cache
    icu/58.3@inseinc/stable:4803f0b3235c36c19a969902b9cf47ed10cdec17 - Cache
    libb64/1.2.1-1@inseinc/stable:d3fb7beef1deae181d55400b6e03fa5d5d5cf8cc - Cache
    mpir/2.7.0-2@inseinc/stable:d3fb7beef1deae181d55400b6e03fa5d5d5cf8cc - Cache
    tinyxml/2.6.2@inseinc/stable:d3fb7beef1deae181d55400b6e03fa5d5d5cf8cc - Cache
    xvid/1.3.7@inseinc/stable:d3fb7beef1deae181d55400b6e03fa5d5d5cf8cc - Cache
    zlib/1.2.11@conan/stable:d3fb7beef1deae181d55400b6e03fa5d5d5cf8cc - Cache
    package-a/none@local/edit:cb989ff2423724a8585c9b30595c6898909e5b6a - Editable
Build requirements
    crc-calculator/1.0.1@inseinc/stable from 'inseinc' - Cache
Build requirements packages
    crc-calculator/1.0.1@inseinc/stable:11e6a84a7894f41df553e7c92534c3bf26896802 - Cache

Cross-build from 'Windows:x86_64' to 'Windows:x86'
Installing (downloading, building) binaries...
bzip2/1.0.8@conan/stable: Already installed!
crc-calculator/1.0.1@inseinc/stable: Already installed!
cryptopp/5.6.5@inseinc/stable: Already installed!
directx/9.0.0-2@inseinc/stable: Already installed!
icu/58.3@inseinc/stable: Already installed!
libb64/1.2.1-1@inseinc/stable: Already installed!
mpir/2.7.0-2@inseinc/stable: Already installed!
tinyxml/2.6.2@inseinc/stable: Already installed!
xvid/1.3.7@inseinc/stable: Already installed!
zlib/1.2.11@conan/stable: Already installed!
boost/1.71.0@conan/stable: Already installed!
boost/1.71.0@conan/stable: LIBRARIES: ['libboost_wave', 'libboost_container', 'libboost_contract', 'libboost_exception', 'libboost_graph', 'libboost_iostreams', 'libboost_locale', 'libboost_log', 'libboost_program_options', 'libboost_random', 'libboost_regex', 
'libboost_serialization', 'libboost_wserialization', 'libboost_coroutine', 'libboost_fiber', 'libboost_context', 'libboost_timer', 'libboost_thread', 'libboost_chrono', 'libboost_date_time', 'libboost_atomic', 'libboost_filesystem', 'libboost_system', 'libboost_type_erasure', 'libboost_log_setup', 'libboost_math_c99', 'libboost_math_c99f', 'libboost_math_c99l', 'libboost_math_tr1', 'libboost_math_tr1f', 'libboost_math_tr1l', 'libboost_stacktrace_noop', 'libboost_stacktrace_windbg', 'libboost_stacktrace_windbg_cached', 'libboost_unit_test_framework']
boost/1.71.0@conan/stable: Package folder: d:\.conan\a8b182\1
boost/1.71.0@conan/stable: Disabled magic autolinking (smart and magic decisions)
package-a/none@local/edit: Applying build-requirement: crc-calculator/1.0.1@inseinc/stable
package-a/none@local/edit: Rewriting files of editable package 'package-a' at 'D:\Git\Frameworks\Library\build/package_a/generated'
package-a/none@local/edit: Generator txt created conanbuildinfo.txt
package-a/none@local/edit: Calling generate()
package-a/none@local/edit: WARN: Using the new toolchains and generators without specifying a build profile (e.g: -pr:b=default) is discouraged and might cause failures and unexpected behavior
package-a/none@local/edit: WARN: Using the new toolchains and generators without specifying a build profile (e.g: -pr:b=default) is discouraged and might cause failures and unexpected behavior
package-a/none@local/edit: MSBuildToolchain created conantoolchain_release_v141_win32.props
package-a/none@local/edit: MSBuildToolchain writing conantoolchain.props
INFO  :runners.py     [87]: Calling command: "C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe" -format json -legacy -nologo > "C:\Users\KAREL~1.DEH\AppData\Local\Temp\tmp8bs9qqji\output" [2022-11-13 20:56:32,736]
INFO  :runners.py     [90]: Return code: 0 [2022-11-13 20:56:32,822]
INFO  :runners.py     [98]: Output: in file:[
                              {
                                "instanceId": "15c7e559",
                                "installDate": "2021-08-03T15:01:39Z",
                                "installationName": "VisualStudio/15.9.51+33027.88",
                                "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional",
                                "installationVersion": "15.9.33027.88",
                                "productId": "Microsoft.VisualStudio.Product.Professional",
                                "productPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\Common7\\IDE\\devenv.exe",
                                "state": 4294967295,
                                "isComplete": true,
                                "isLaunchable": true,
                                "isPrerelease": false,
                                "isRebootRequired": false,
                                "displayName": "Visual Studio Professional 2017",
                                "description": "Professional developer tools and services for small teams",
                                "channelId": "VisualStudio.15.Release",
                                "channelPath": "C:\\Users\\xyz\\AppData\\Local\\Microsoft\\VisualStudio\\Packages\\_Channels\\4CB340F5\\install_catalog.json",
                                "channelUri": "https://aka.ms/vs/15/release/channel",
                                "enginePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\resources\\app\\ServiceHub\\Services\\Microsoft.VisualStudio.Setup.Service",
                                "installChannelUri": "\\\\WSUSBUR\\vs2k17$\\ChannelManifest.json",
                                "layoutPath": "\\\\WSUSBUR\\vs2k17$",
                                "releaseNotes": "https://go.microsoft.com/fwlink/?LinkId=660692#15.9.51",
                                "thirdPartyNotices": "https://go.microsoft.com/fwlink/?LinkId=660708",
                                "updateDate": "2022-11-12T18:55:44.7601719Z",
                                "catalog": {
                                  "buildBranch": "d15.9",
                                  "buildVersion": "15.9.33027.88",
                                  "id": "VisualStudio/15.9.51+33027.88",
                                  "localBuild": "build-lab",
                                  "manifestName": "VisualStudio",
                                  "manifestType": "installer",
                                  "productDisplayVersion": "15.9.51",
                                  "productLine": "Dev15",
                                  "productLineVersion": "2017",
                                  "productMilestone": "RTW",
                                  "productMilestoneIsPreRelease": "False",
                                  "productName": "Visual Studio",
                                  "productPatchVersion": "51",
                                  "productPreReleaseMilestoneSuffix": "1.0",
                                  "productRelease": "RTW",
                                  "productSemanticVersion": "15.9.51+33027.88",
                                  "requiredEngineVersion": "1.18.1063.29791"
                                },
                                "properties": {
                                  "campaignId": "",
                                  "channelManifestId": "VisualStudio.15.Release/15.9.51+33027.88",
                                  "nickname": "",
                                  "setupEngineFilePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vs_installershell.exe"
                                }
                              },
                              {
                                "instanceId": "VisualStudio.14.0",
                                "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\",
                                "installationVersion": "14.0"
                              },
                              {
                                "instanceId": "VisualStudio.12.0",
                                "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\",
                                "installationVersion": "12.0"
                              },
                              {
                                "instanceId": "VisualStudio.10.0",
                                "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\",
                                "installationVersion": "10.0"
                              }
                            ]

                            stdout: None
                            stderr:b'' [2022-11-13 20:56:32,833]
INFO  :runners.py     [87]: Calling command: "C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe" -format json -products * -nologo > "C:\Users\KAREL~1.DEH\AppData\Local\Temp\tmpe67pfa_0\output" [2022-11-13 20:56:32,848]
INFO  :runners.py     [90]: Return code: 0 [2022-11-13 20:56:32,965]
INFO  :runners.py     [98]: Output: in file:[
                              {
                                "instanceId": "15c7e559",
                                "installDate": "2021-08-03T15:01:39Z",
                                "installationName": "VisualStudio/15.9.51+33027.88",
                                "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional",
                                "installationVersion": "15.9.33027.88",
                                "productId": "Microsoft.VisualStudio.Product.Professional",
                                "productPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\Common7\\IDE\\devenv.exe",
                                "state": 4294967295,
                                "isComplete": true,
                                "isLaunchable": true,
                                "isPrerelease": false,
                                "isRebootRequired": false,
                                "displayName": "Visual Studio Professional 2017",
                                "description": "Professional developer tools and services for small teams",
                                "channelId": "VisualStudio.15.Release",
                                "channelPath": "C:\\Users\\xyz\\AppData\\Local\\Microsoft\\VisualStudio\\Packages\\_Channels\\4CB340F5\\install_catalog.json",
                                "channelUri": "https://aka.ms/vs/15/release/channel",
                                "enginePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\resources\\app\\ServiceHub\\Services\\Microsoft.VisualStudio.Setup.Service",
                                "installChannelUri": "\\\\WSUSBUR\\vs2k17$\\ChannelManifest.json",
                                "layoutPath": "\\\\WSUSBUR\\vs2k17$",
                                "releaseNotes": "https://go.microsoft.com/fwlink/?LinkId=660692#15.9.51",
                                "thirdPartyNotices": "https://go.microsoft.com/fwlink/?LinkId=660708",
                                "updateDate": "2022-11-12T18:55:44.7601719Z",
                                "catalog": {
                                  "buildBranch": "d15.9",
                                  "buildVersion": "15.9.33027.88",
                                  "id": "VisualStudio/15.9.51+33027.88",
                                  "localBuild": "build-lab",
                                  "manifestName": "VisualStudio",
                                  "manifestType": "installer",
                                  "productDisplayVersion": "15.9.51",
                                  "productLine": "Dev15",
                                  "productLineVersion": "2017",
                                  "productMilestone": "RTW",
                                  "productMilestoneIsPreRelease": "False",
                                  "productName": "Visual Studio",
                                  "productPatchVersion": "51",
                                  "productPreReleaseMilestoneSuffix": "1.0",
                                  "productRelease": "RTW",
                                  "productSemanticVersion": "15.9.51+33027.88",
                                  "requiredEngineVersion": "1.18.1063.29791"
                                },
                                "properties": {
                                  "campaignId": "",
                                  "channelManifestId": "VisualStudio.15.Release/15.9.51+33027.88",
                                  "nickname": "",
                                  "setupEngineFilePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vs_installershell.exe"
                                }
                              }
                            ]

                            stdout: None
                            stderr:b'' [2022-11-13 20:56:32,976]
package-a/none@local/edit: Aggregating env generators
package-a/none@local/edit: Generated toolchain
package-a/none@local/edit: Generated conan.lock
package-a/none@local/edit imports(): Copied 1 '.exe' file: CrcCalculator.exe
package-a/none@local/edit: Copied 1 '.exe' file: CrcCalculator.exe
conanfile.py (package_b/12.2.25-alpha.9+git.48274b0): WARN: Using the new toolchains and generators without specifying a build profile (e.g: -pr:b=default) is discouraged and might cause failures and unexpected behavior
conanfile.py (package_b/12.2.25-alpha.9+git.48274b0): Generator 'MSBuildDeps' calling 'generate()'
conanfile.py (package_b/12.2.25-alpha.9+git.48274b0): ERROR: Traceback (most recent call last):
  File "conans\client\generators\__init__.py", line 181, in write_generators
  File "conan\tools\microsoft\msbuilddeps.py", line 109, in generate
  File "conan\tools\microsoft\msbuilddeps.py", line 334, in _content
  File "conan\tools\microsoft\msbuilddeps.py", line 319, in _package_props_files
  File "conan\tools\microsoft\msbuilddeps.py", line 167, in _vars_props_file
  File "conan\tools\microsoft\msbuilddeps.py", line 153, in escape_path
AttributeError: 'NoneType' object has no attribute 'replace'

ERROR: Error in generator 'MSBuildDeps': 'NoneType' object has no attribute 'replace'
memsharded commented 2 years ago

Whilst I was waiting for your response, I figured out the cause of the issue was a name clashing with something Conan uses and I was typing the object to be a list. the object was called "dependencies". This is why the error occurred.

Good catch. Yes, there are some rules in the docs in https://docs.conan.io/en/latest/reference/conanfile.html to avoid this type of conflicts, please use your own attributes and methods as _mymethod() and _myattribute, as the public ones and _conan_xxx are reserved for Conan.

Thanks for sharing your conanfiles. I have checked it, and indeed it seems a bug in the MSBuildDeps generator when the dependencies are in editable mode. Related comment in this issue: https://github.com/conan-io/conan/issues/12310#issuecomment-1300512984

Let's try to fix it for next release.

I'd also like to share some general feedback that might help to start being prepared for 2.0 (that is getting very close):

There are some guidelines in https://docs.conan.io/en/latest/conan_v2.html, it would be great to keep an eye on them, we have done a ton of effort to backport things to 1.X, so the upgrade to 2.0 could be much smoother.

memsharded commented 2 years ago

Trying a fix in https://github.com/conan-io/conan/pull/12529, for next 1.55

KingKiller100 commented 2 years ago

Wonderful. I will await v1.55.x for the fix