[question] How could I debug the conanfile.py stages (e.g. via pdb / PyCharm / idle / wing ide / etc)

[leha-bot]

I want to debug the Python side of conan recipe and check the particular stages of my recipe. How could I start debugging or attaching to conan create / conan package / another stage using the Python tooling? (I thought that this information exists in this section of the Conan's documentation, but didn't found it :( )

Could you please also add this answer to the documentation?

Thanks in advance.

• [x] I've read the CONTRIBUTING guide.

[zink-chimaera]

Starting via Python python -m conans.conan <args> (e.g. python -m conans.conan create .) does not enable you to debug (why?).

With Pycharm-Professional you can insert your debugger into the running instance with

import pydevd_pycharm
pydevd_pycharm.settrace('localhost', port=5678, stdoutToServer=True, stderrToServer=True)

However regular debugging support is definetly needed IMO. Are there any statements or updates on this matter? I coudn't find anything either.

Edit: Fixed command

[Lonenso]

I also met this problem. I can set pdb.set_trace() in configure()/package() to debug conan install/export-pkg, but I can not use pdb.set_trace to step into package_info()/package_id()

oh, sorry, the "package_info" function will be invoked when the package was installed. let me provide an example, package BB depends on AA.

from conans import ConanFile, tools
import pdb

class AA(ConanFile):
    name = "AA"
    version = "0.1"
    settings = "os", "compiler", "build_type", "arch"
    description = "package AA"

    def configure(self):
        self.output.info("configure...")

    def requirements(self):
        self.output.info("requirements...")
        self.requires("poco/1.9.4")

    def build(self):
        self.output.info("build...")

    def package(self):
        self.output.info("package...")

    def package_id(self):
        self.output.info("package_id...")

    def package_info(self):
        self.output.info("package_info...")

❯ conan export-pkg . AA/0.1@local/stable
Exporting package recipe
AA/0.1@local/stable: A new conanfile.py version was exported
AA/0.1@local/stable: Folder: /Users/peter.xie/.conan/data/AA/0.1/local/stable/export
AA/0.1@local/stable: Using the exported files summary hash as the recipe revision: f45cb65c4ddebd267e4fa395ac07da17
AA/0.1@local/stable: Exported revision: f45cb65c4ddebd267e4fa395ac07da17
AA/0.1@local/stable: configure...
AA/0.1@local/stable: requirements...
AA/0.1@local/stable: package_id...
AA/0.1@local/stable: Forced build from source
Packaging to 3b118aefdf510348d4dd9e77d77ecbcff47d968b
AA/0.1@local/stable: Generating the package
AA/0.1@local/stable: Package folder /Users/peter.xie/.conan/data/AA/0.1/local/stable/package/3b118aefdf510348d4dd9e77d77ecbcff47d968b
AA/0.1@local/stable: Calling package()
AA/0.1@local/stable: package...
AA/0.1@local/stable package(): WARN: No files in this package!
AA/0.1@local/stable: Package '3b118aefdf510348d4dd9e77d77ecbcff47d968b' created
AA/0.1@local/stable: Created package revision 534f311e260f80a30b27379a20ff7d6c
❯ conan upload AA/0.1@local/stable -r=local

as you can see, there's no package_info output. package_info won't be invoked.

from conans import ConanFile, tools
import pdb

class BB(ConanFile):
    name = "BB"
    version = "0.1"
    settings = "os", "compiler", "build_type", "arch"
    description = "package BB"

    def configure(self):
        self.settings.os.version = 10.0

    def requirements(self):
        self.output.info("requirements...")
        self.requires("AA/0.1@local/stable")

    def build(self):
        self.output.info("build...")

    def package(self):
        self.output.info("package...")

    def package_id(self):
        self.output.info("package_id...")

    def package_info(self):
        self.output.info("package_info...")

❯ conan install .
Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=apple-clang
compiler.libcxx=libc++
compiler.version=12.0
os=Macos
os_build=Macos
[options]
[build_requires]
[env]

conanfile.py (BB/0.1): configure...
conanfile.py (BB/0.1): requirements...
AA/0.1@local/stable: configure...
AA/0.1@local/stable: requirements...
AA/0.1@local/stable: package_id...
conanfile.py (BB/0.1): package_id...
conanfile.py (BB/0.1): Installing package
Requirements
    AA/0.1@local/stable from 'local' - Cache
    bzip2/1.0.8 from 'conan-center' - Cache
    expat/2.4.1 from 'conan-center' - Cache
    openssl/1.1.1k from 'conan-center' - Cache
    pcre/8.44 from 'conan-center' - Cache
    poco/1.9.4 from 'conan-center' - Cache
    sqlite3/3.35.5 from 'conan-center' - Cache
    zlib/1.2.11 from 'conan-center' - Cache
Packages
    AA/0.1@local/stable:3b118aefdf510348d4dd9e77d77ecbcff47d968b - Cache
    bzip2/1.0.8:75fe66fbfe61230e9fcbb361f13b6b3baa483e0a - Cache
    expat/2.4.1:60570b031d430e6f0038a99498471dea196b9d22 - Cache
    openssl/1.1.1k:647afeb69d3b0a2d3d316e80b24d38c714cc6900 - Cache
    pcre/8.44:94834772cc1442ba83e7c43a04d21e0246867136 - Cache
    poco/1.9.4:580a2e69ddec0f0dad838de253ed1be0b0d4f775 - Cache
    sqlite3/3.35.5:264f180b28ff91ffa6d05e858eb8e55cc6c1a073 - Cache
    zlib/1.2.11:647afeb69d3b0a2d3d316e80b24d38c714cc6900 - Cache

Installing (downloading, building) binaries...
bzip2/1.0.8: Already installed!
bzip2/1.0.8: Appending PATH environment variable: /Users/peter.xie/.conan/data/bzip2/1.0.8/_/_/package/75fe66fbfe61230e9fcbb361f13b6b3baa483e0a/bin
expat/2.4.1: Already installed!
openssl/1.1.1k: Already installed!
sqlite3/3.35.5: Already installed!
sqlite3/3.35.5: Appending PATH env var with : /Users/peter.xie/.conan/data/sqlite3/3.35.5/_/_/package/264f180b28ff91ffa6d05e858eb8e55cc6c1a073/bin
zlib/1.2.11: Already installed!
pcre/8.44: Already installed!
pcre/8.44: Appending PATH environment variable: /Users/peter.xie/.conan/data/pcre/8.44/_/_/package/94834772cc1442ba83e7c43a04d21e0246867136/bin
poco/1.9.4: Already installed!
AA/0.1@local/stable: Already installed!
AA/0.1@local/stable: package_info...  *** see here ***
conanfile.py (BB/0.1): Generator txt created conanbuildinfo.txt
conanfile.py (BB/0.1): Generated conaninfo.txt
conanfile.py (BB/0.1): Generated graphinfo

you could use pdb.set_trace like self.output...

reference: https://docs.conan.io/en/latest/reference/conanfile/other.html#output-and-running https://docs.conan.io/en/latest/reference/conanfile/methods.html#package-info

[GiugAles]

Since the title contains "etc" I would like to point out the possibility to debug conanfile.py stages with VSCode. Though I did not test it with other IDEs, which is actually why I am ending up here, my experiences with VSCode in this matter are quite good. You just need to create a launch.json as described in https://code.visualstudio.com/docs/editor/debugging. Than point to the conan executable. In the end, it looks more less like this:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "conan install package",
            "type": "python",
            "request": "launch",
            "program": "/usr/local/bin/conan",
            "console": "integratedTerminal",
            "args": ["install", ".", "-o", "example_option=True"]
        },
    ]
}

Then you can start the debugger. Works also when debugging in the cache.

[leha-bot]

Thanks everyone for answers, I also found that article with vscode example. It's though very sad that docs doesn't have any info how to properly debug a recipe (maybe it already exist, as I didn't inspect docs?)

Closing this question as "solved", but thinking that Conan docs should document these practices.

[memsharded]

Lets re-open and move it to the docs repo, even if the backlog is large and it is unlikely to be high priority, lets keep it open so eventually it could be added to the docs. Thanks for the feedback.

[zink-chimaera]

I'm calling conan from within python which enables you to debug most of it. Maybe something similar can be added in order to make it easier for people familiar with Python to help them help themselves?

from conans.client.command import main as conan_command

conan_command("conan create -pr ...".split())

Or, because conan_command will throw a SystemExit once its done so if you want do do more than that, you need to catch it and check for errors yourself, e.g.:

def conan(command: str):
    print(f"conan {command}")
    try:
        conan_command(command.split())
    except SystemExit as e:
        if e.code != 0:
            print(f"{command.split()[0]} failed")
            raise e