ImportError – name clash when debugging MacOS Kernel with LLDB

[golddranks]

I'm trying to remote debug MacOS kernel with LLDB. I have Voltron and Apple's Kernel Debug Kit installed.

In LLDB, first I create the target:

target create /Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel

After that, once I set it to load scripts:

settings set target.load-script-from-symbol-file true

Then, it crashes with

error: module importing failed: Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros/xnu.py", line 1286, in <module>
    from misc import *
  File "/Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros/misc.py", line 8, in <module>
    from scheduler import *
  File "/Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros/scheduler.py", line 1035, in <module>
    from kevent import GetKnoteKqueue
  File "/Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros/kevent.py", line 3, in <module>
    from memory import vm_unpack_pointer
ImportError: cannot import name 'vm_unpack_pointer' from 'memory' (/Users/kon/Library/Python/3.8/lib/python/site-packages/voltron/plugins/api/memory.py)

The problem seems to be that a file /Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros/kevent.py tries to import stuff from memory, where the intended file for that is /Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros/memory.py

However, for some reason, it actually accesses Voltron's memory.py: /Users/me/Library/Python/3.8/lib/python/site-packages/voltron/plugins/api/memory.py

I am not very well-versed in Pythons modules and import rules, and especially I don't know how LLDB loads scripts, but to me, this seems like something that is not normally supposed to happen. Is this just unfortunate name clash, or is something broken in my system?

[akx]

This seems like 'voltron/plugins/api' has ended up in the Python interpreter's sys.path, so all absolute imports end up being imported from there.

However, looking at the other imports in the traceback, it seems like lldbmacros expects itself to be the first in path anyway (otherwise the other imports would be from lldbmacros.scheduler ... or from .scheduler ... (or it's Python 2, which evidently it isn't).

IOW, print(sys.path) – what does it look like?

[golddranks]

So I snuck a print statement in the start of xnu.py which was at the bottom of the callstack mentioned above. It says:

['/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Versions/A',
'/Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros',
'/Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python',
'/Users/kon/Library/Python/3.8/lib/python/site-packages/voltron',
'/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python38.zip',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/lib-dynload',
'/Users/kon/Library/Python/3.8/lib/python/site-packages',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages',
'.',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload']

For reference, here's sys.path printed from running python3 normally (I have only the Python installed by XCode CLI tools currently linked in my system; LLDB uses that.):

['',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python38.zip',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/lib-dynload',
'/Users/kon/Library/Python/3.8/lib/python/site-packages',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages']

So, LLDB (or some of the scripts?) seems to tweak sys.path?

Also, it should be noted that the KDK lldbmacros are converted from Python 2 to Python 3 by myself:

sudo 2to3 -w /Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros/

...this is because LLDB doesn't seem to support Python 2 anymore, but the KDKs for older kernels are just that.

It's slightly cursed, but it used to somehow work before I installed Voltron. Checking the code, it seems that 2to3 doesn't convert the relative imports like from .scheduler import *, they are like from scheduler import *.

[akx]

Yeah,

/Users/kon/Library/Python/3.8/lib/python/site-packages/voltron'

should absolutely not be on sys.path by itself.

It's enough that site-packages is, since that will allow you to import voltron... or from voltron... import ..., but having the voltron package's directory there too will (and does) cause chaos.

If you can find out why it's there, that'd be the next step.

• probably an invocation related to sys.path somewhere, likely in Voltron itself
• Another option is that there's a .pth file that refers the voltron package directory; the site machinery in Python scans those and puts them on sys.path.
• https://github.com/snare/voltron/blob/d9fef0bb4073d434c54415d55a1b5da92cb84696/install.sh#L271-L274 seems interesting too; maybe look at your lldb init file to see if there's the append invocation there

[golddranks]

My $HOME/.lldbinit contains just

command script import /Users/kon/Library/Python/3.8/lib/python/site-packages/voltron/entry.py

and manually tweaking /Users/kon/Library/Python/3.8/lib/python/site-packages/voltron/entry.py to print sys.path at start yields:

['/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Versions/A',
'/Users/kon/Library/Python/3.8/lib/python/site-packages/voltron',
'/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python38.zip',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/lib-dynload',
'/Users/kon/Library/Python/3.8/lib/python/site-packages',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages',
'.']

so the voltron path is already there before the voltron entry script is called.

Also, no .pth files in site-packages or the voltron subdir. And quick grepping the package doesn't reveal anything else related to sys.path but entry.py adding some stuff here: https://github.com/snare/voltron/blob/master/voltron/entry.py#L31 , but that's after I printed the sys.path that already contained the voltron path.

[akx]

For one, I don't think that linked segment in voltron/entry.py should be there at all in 2022, since it's all just Python 2.7 fluff.

What if you do command script import sys;print("Honk!", sys.path) in .lldbinit?

[golddranks]

We're getting closer! /Users/me/entry.py is an one-liner that prints the current sys.path. .lldbinit:

command script import sys;print(sys.path)
command script import /Users/me/entry.py

['/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Versions/A',
'/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python38.zip',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/lib-dynload',
'/Users/me/Library/Python/3.8/lib/python/site-packages',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages',
'.']
my entry:  ['/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Versions/A',
'/Users/me',
'/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python38.zip',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/lib-dynload',
'/Users/me/Library/Python/3.8/lib/python/site-packages',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages',
'.']

So, it seems like the culprit is LLDB...?

[akx]

Ohhhhh, now I get it! Yep - so clearly command script import foo/baz.py adds foo to sys.path, then imports baz. (Makes sense.)

command script import /Users/kon/Library/Python/3.8/lib/python/site-packages/voltron/entry.py
command script 'import sys; sys.path[:] = [p for p in sys.path if not p.endswith("/voltron")]'

or similar should fix that up... 😁

[golddranks]

That would work, if it would accept any kind of python statements, but seems the expression you can use there has some rather haphazard limitations. First of all, import is part of the LLDB command, and it is required. After that, it seems to allow only one space directly after, and no whitespace after that. Clearly the user is meant just to pass in a path there, but it doesn't bother validating the input properly. I was able to delete the path with

command script import sys;sys.path=sys.path[:1]+sys.path[2:]

But that depends on the order of the path and feels like it's from a code golf competition. I also tried calling a script that deletes the path, but that doesn't seems to affect the path outside of the said script.

It would help if voltron could be imported not as a script file entry.py, but as a package. I tried to fix it by creating a new wrapper:

mkdir /Users/kon/Library/Python/3.8/lib/python/site-packages/voltron_lldb
echo "from voltron import entry" > /Users/kon/Library/Python/3.8/lib/python/site-packages/voltron_lldb/__init__.py
echo "command script import /Users/kon/Library/Python/3.8/lib/python/site-packages/voltron_lldb" > $HOME/.lldbinit

And it ALMOST works, no /Users/kon/Library/Python/3.8/lib/python/site-packages/voltron to be seen!

command script import sys;print(sys.path)
['/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Versions/A',
'/Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros',
'/Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python',
'/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python38.zip',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/lib-dynload',
'/Users/kon/Library/Python/3.8/lib/python/site-packages',
'/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages',
'.',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload']

But no cigar:

(lldb) target create /Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel
(lldb) settings set target.load-script-from-symbol-file true

error: module importing failed: Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros/xnu.py", line 1287, in <module>
    from misc import *
  File "/Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros/misc.py", line 8, in <module>
    from scheduler import *
  File "/Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros/scheduler.py", line 1035, in <module>
    from kevent import GetKnoteKqueue
  File "/Library/Developer/KDKs/KDK_11.6.5_20G527.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/Python/lldbmacros/kevent.py", line 3, in <module>
    from memory import vm_unpack_pointer
ImportError: cannot import name 'vm_unpack_pointer' from 'memory' (/Users/kon/Library/Python/3.8/lib/python/site-packages/voltron/plugins/api/memory.py)

WHY? \:D/ I have no idea. (The same thing happens both with the wrapper, and the path deletion method.)

[golddranks]

Could it be that it caches something? And now that you mention it, how does it even find memory.py from a subdirectory of voltron, plugins/api/memory.py? Btw. I renamed that file to voltron_memory.py, and it found another one from plugins/view/memory.py. KDK started working only after renaming those two...

[akx]

Oh, right, yeah, you'd maybe also need del sys.modules['memory'] to clear Python's memory of having imported that module as memory...

[day0n]

There is someone upadate?