Gdb tab auto complete broken by `import readline`

[PandaWill]

Description

Gdb tab autocomplete is broken. It appears that environ["PWNLIB_NOTERM"]=1 is the reason why.

It starts working if I remove this line: https://github.com/pwndbg/pwndbg/blob/bf091df52781de81b7d4662fc65ce18d11aeecb8/gdbinit.py#L59

Steps to reproduce

Open Gdb, type st<TAB>

We expect it to show the options of "start", "step", etc. We get a literal tab.

My setup

pwndbg> version Gdb 12.1 Python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] Pwndbg: 2024.02.14 build: ffc09e57 Capstone: 5.0.1280 Unicorn: 2.0.1

[B1N4RY-P4R45173]

I am still very new to this project. But I have noticed two things: 1) There are multiple commands starting with st so it cannot autocomplete. I am assuming u are referring to autocomplete suggestion?? I noticed that pressing the tab again displays all suggestions. Maybe a bug?? idk....

2) If you were actually referring to autocomplete then try sto and hitting tab key. It should complete it to the word stop. Check if that works.

[PandaWill]

Sorry, yes that's what I mean suggestions. I don't get the suggestions or autocomplete though.

Unless I comment out the environ["PWNLIB_NOTERM"]=1 line (or set it to zero).

https://github.com/pwndbg/pwndbg/blob/bf091df52781de81b7d4662fc65ce18d11aeecb8/gdbinit.py#L59

[B1N4RY-P4R45173]

I tried changing PWNLIB_NOTERM to 0. It didn't seem to invoke any issues at the moment but according to

https://github.com/pwndbg/pwndbg/blob/bf091df52781de81b7d4662fc65ce18d11aeecb8/docs/env_vars.md?plain=1#L17

U might face issues with decencies that use pwntools. And did it work fine before https://github.com/pwndbg/pwndbg/commit/ffc09e57f3b870f99f52815b983ef678bf69636f ?

[disconnect3d]

@PandaWill The autocompletion works for me no matter of the PWNLIB_NOTERM setting.

What distro, window manager and terminal do you use?

[PandaWill]

Sorry, my initial diagnosis of this was misplaced.

I went on a wild ride of debugging, hacking out lots of code to find out what was breaking it.

I narrowed it down to the import pdb inside construct, inside pyelftools.

It's because I've been using pdb++ (drop in replacement for pdb).

I'll take the issue up with those folks, I guess they need a similar environmental variable for: "I don't own the terminal".

[disconnect3d]

Thanks for looking into it! Please link the issue here once you open it on their side :)

On Tue, 25 Jun 2024 at 12:00, Will @.***> wrote:

Sorry, my initial diagnosis of this was misplaced.

I went on a wild ride of debugging, hacking out lots of code to find out what was breaking it.

I narrowed it down to the import pdb inside construct, inside pyelftools.

It's because I've been using pdb++ (drop in replacement for pdb).

I'll take the issue up with those folks, I guess they need a similar environmental variable for: "I don't own the terminal".

— Reply to this email directly, view it on GitHub https://github.com/pwndbg/pwndbg/issues/2232#issuecomment-2188486379, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACMLWCXJRW64FVSRX5LBFH3ZJE5U5AVCNFSM6AAAAABJISGUMSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBYGQ4DMMZXHE . You are receiving this because you commented.Message ID: @.***>

[PandaWill]

They actually already have an option for this but it doesn't fix it because autocomplete is already broken by their import statements!

Namely fancycompleter, which imports rlcompleter, which imports readline.

Importing readline from the standard library is enough to break Gdb autocomplete.

You can see this by adding import readline to the gdbinit.py and it breaks autocomplete.

Presumably this is a horrible conflict between the CPython's and Gdb's use of readline. I have no idea how to fix it.

The same problem can be encountered if you use pwndbg exception debugging with ipdb installed: https://github.com/pwndbg/pwndbg/blob/83cc8c57cf87409a4e7748abefd56bb31b4d9cd9/pwndbg/exception.py#L13

Although I can't see where ipdb would be importing readline, except within pdb itself: https://github.com/python/cpython/blob/main/Lib/pdb.py#L324

Regular pdb (not ipdb) doesn't break it, neither does using IPython.embed(). I'm not sure why.

[Aplet123]

Python's readline module seems to do a lot of stuff on initialization (yuck) like rebinding tab to insert a literal tab, and ESC-TAB to use their completion function instead:

https://github.com/python/cpython/blob/3ec719fabf936ea7a012a76445b860759155de86/Modules/readline.c#L1296-L1299

[PandaWill]

There is support in readline for saving and restoring state https://git.savannah.gnu.org/cgit/readline.git/tree/readline.c#n1459

But cpython is doing none of this, it just assumes it's the sole user of readline and blindly re-initializes everything!

I feel like this needs to be fixed in cpython! Any other project that (like Gdb) uses readline and embeds python will run into the same issue.

[Aplet123]

I think a hacky solution could just be adding python 'import sys; sys.modules["readline"] = None' (or some more intelligent mock than None) into the start of .gdbinit to just prevent the module from loading, but this is pretty awful.

@PandaWill Thoughts on closing the issue because it's not really a pwndbg issue?

[PandaWill]

I/we could add a test case that just imports pwndbg and checks for readline in sys.modules?

It'd be nice to have something to make sure we notice if it's accidentally broken in the future. Especially since it can be trivially broken by something deep down the dependency chain.

Also it would save somebody else from going down this rabbit hole in the future!