Include debug symbols in MacOS for 3.9-3.11

[pablogsal]

Hi 👋 ,

I'm Pablo Galindo, the 3.10 and 3.11 release manager of CPython. I am opening this issue because I would like to discuss the possibility of automatically building dSYM files in interpreter builds in macOS so users can leverage native debuggers.

Check this issue for a summary of the problem:

https://github.com/python/cpython/issues/95973

The summary is that starting with Python 3.12 we in the core team are going to include a new option (--with-dsymutil) that automatically packages debug information in interpreter builds for macOS so users can use lldb and other debuggers to debug Python crashes and use native profilers.

For 3.12, pyenv would just need to add --with-dsymutil to the list of configure flags in macOS but for previous existing Python versions this needs to be done manually as part of the build process in pyenv. This can be easily done by calling dsymutil over the installed final Python executable and all the shared libraries when the temporary .o files are still available somewhere on disk.

Thanks for considering the feature and thank you for your fantastic work!

Please feel free to ask me any questions if something is not clear or if you have any questions.

[pablogsal]

Thank you very much for the quick response!

One small detail to clarify: I can see that the attached PR does the requested changes for 3.12 but there are no PRs for previous versions. Do I understand correctly that that means that there is then no intention to manually implement the feature for previous versions?

[native-api]

You said, in this very ticket, that this feature is only implemented since 3.12.

So not only shoudn't we apply it elsewhere, we can't, as that would cause an unrecognized switch error.

[pablogsal]

I said:

For 3.12, pyenv would just need to add --with-dsymutil to the list of configure flags in macOS but for previous existing Python versions this needs to be done manually as part of the build process in pyenv. This can be easily done by calling dsymutil over the installed final Python executable and all the shared libraries when the temporary .o files are still available somewhere on disk.

This means that for 3.12 we can use the features directly from cpython but as my paragraph say for previous existing versions it needs to be manually implemented on the build process.

Quoting myself yet again:

This can be easily done by calling dsymutil over the installed final Python executable and all the shared libraries when the temporary .o files are still available somewhere on disk.

[native-api]

Okay, I admit I skimmed through that part 'cuz I didn't intend to do more myself than supporting the switch (and didn't expect that you would expect more).

For anything more involved, I feel you're in a better position to provide a PR, or at least logic for it, since you're more familiar with CPython's build process, what exactly needs to be done and at what points. For starters, AFAICS, this results in something called "dSYM package" that would also need to be placed somewhere, probably also with some very specific name for the debugger to automagically match it to an executable.

This also raises other questions:

1) Since to debug, one would needs sources -- it probably only makes sense to do that if the user has requested to retain sources? 2) How far back should we go? I guess dsymutil is a fairly recent addition since this has only surfaced now. We have some really ancient Python versions here.

[pablogsal]

Thanks for your quick response and your comments!

Since to debug, one would needs sources -- it probably only makes sense to do that if the user has requested to retain sources?

Not in my opinion , ideally this should be done by default, otherwise debugging any Python crash or native extension won't be possible because by default nothing will have symbols and will be too late once problems appear. This will make life more difficult to users and package maintainers. I would recommend to provide an opt-out, not an opt-in.

How far back should we go? I guess dsymutil is a fairly recent addition since this has only surfaced now. We have some really ancient Python versions here.

I would recommend only doing 3.9, 3.10 and 3.11 as these are the current supported versions upstream.

Also note that this problem has existed for a long time since macOS never packaged or produced debugging information by default. The reasons this surfaced now is because we are trying to address it directory upstream and find ways users can benefit from retroactive fixes, that's why I opened this issue

[pablogsal]

I will try to produce a Pull Request for adding this when building 3.11 and we can discuss from there.

[berndverst]

@pablogsal since 3.7 and 3.8 are not EOL yet, I think it would also help to include the debug symbols for 3.7+

Keep in mind that many Enterprise users only upgrade to new minor versions if their existing version is EOL.

[pablogsal]

I'm closing this issue since I won't have time soon to make the aforementioned PRs.

[jevinskie]

Just FYI I did this with pyenv 3.10.13 to try to get cypython-lldb (which only supports up to 3.10) working. The cypython-lldb scripts sort of worked but it was definitely nice to get real debug info and back traces. It’s basically a squash of the initial 3.12 PR commits which I then patched up to fix DSYMUTIL and DSYMUTIL_PATH env-vars from being ignored because macOS ld64 (or is it cctools? I forget) doesn’t handle -fdebug-macros while LLVM 17’s dsymutil seems to handle them OK. You can see what it looks like here.

[pablogsal]

@jevinskie Thanks for the comment! Yeah, what needs to be done is include them in the pyenv build scripts and then activate them by default.

Maybe you can do a PR to contribute this to pyenv?