Support for pyproject.toml is broken, and breaks debugging

[whyscream]

As found in https://github.com/gotcha/ipdb/pull/213, ipdb tries to read configuration from pyproject.toml using a ConfigParser object. However, ConfigParser does not support toml syntax, it supports ini syntax which looks a lot alike, but is not the same.

I'm using the following snippet in my pyproject.toml, and ipdb crashes whenever it is is started:

[tool.black]
line-length = 119
target-version = ['py38']
exclude = '''
/(
    | migrations
)/
'''

I don't have any ipdb-related configuration in pyproject.toml at all.

The traceback:

Traceback (most recent call last):
  File "[...]/env/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "[...]/env/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "[...]/env/lib/python3.8/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "[...]/env/lib/python3.8/site-packages/django/views/generic/base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
  File "[...]/env/lib/python3.8/site-packages/django/views/generic/edit.py", line 133, in get
    return self.render_to_response(self.get_context_data())
  File "[...]/website/views.py", line 43, in get_context_data
    breakpoint()
  File "[...]/env/lib/python3.8/site-packages/ipdb/__main__.py", line 75, in set_trace
    p = _init_pdb(context).set_trace(frame)
  File "[...]/env/lib/python3.8/site-packages/ipdb/__main__.py", line 52, in _init_pdb
    context = os.getenv("IPDB_CONTEXT_SIZE", get_context_from_config())
  File "[...]/env/lib/python3.8/site-packages/ipdb/__main__.py", line 82, in get_context_from_config
    parser = get_config()
  File "[...]/env/lib/python3.8/site-packages/ipdb/__main__.py", line 174, in get_config
    read_func(f)
  File "/usr/lib/python3.8/configparser.py", line 718, in read_file
    self._read(f, source)
  File "/usr/lib/python3.8/configparser.py", line 1113, in _read
    raise e
configparser.ParsingError: Source contains parsing errors: '[...]/pyproject.toml'
    [line 13]: '/(\n'
    [line 15]: ')/\n'
    [line 16]: "'''\n"

I think ipdb should use a toml parser to parse pyproject.toml, or stay away from the file. Note that this change will hurt a lot of people using the combination of ipdb and black.

CC @alexandrebarbaruiva

[alexandrebarbaruiva]

Looking into it right now

[alexandrebarbaruiva]

@whyscream could you test my changes to see if this issue has been solved?

[whyscream]

@alexandrebarbaruiva Thanks for the quick response.

Unfortunately, in your PR stuff still is broken, but in a different way :/ To reproduce:

• Create a venv, activate, and install your branch using pip install ../path/to/ipdb/checkout
• Create a directory with a pyproject,toml file, with above content.
• In the directory with the pyproject.toml, execute the commands:

(env) $ ipython
Python 3.9.2 (default, Feb 20 2021, 20:56:08) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.21.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import ipdb
In [2]: ipdb.set_trace()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-2-823a13fd69ba> in <module>
----> 1 ipdb.set_trace()

~/code/ipdb/ipdb/__main__.py in set_trace(frame, context, cond)
     73     if frame is None:
     74         frame = sys._getframe().f_back
---> 75     p = _init_pdb(context).set_trace(frame)
     76     if p and hasattr(p, 'shell'):
     77         p.shell.restore_sys_module_state()

~/code/ipdb/ipdb/__main__.py in _init_pdb(context, commands)
     50 def _init_pdb(context=None, commands=[]):
     51     if context is None:
---> 52         context = os.getenv("IPDB_CONTEXT_SIZE", get_context_from_config())
     53     try:
     54         p = debugger_cls(context=context)

~/code/ipdb/ipdb/__main__.py in get_context_from_config()
     80 def get_context_from_config():
     81     try:
---> 82         parser = get_config()
     83         return parser.getint("tool.ipdb", "context", fallback=parser.getint("ipdb", "context"))
     84     except (configparser.NoSectionError, configparser.NoOptionError):

~/code/ipdb/ipdb/__main__.py in get_config()
    177                 import toml
    178                 toml_file = toml.load(filepath)
--> 179                 parser["ipdb"] = toml_file["tool"].get("ipdb")
    180             else:
    181                 read_func(ConfigFile(filepath))

/usr/lib/python3.9/configparser.py in __setitem__(self, key, value)
    972         elif key in self._sections:
    973             self._sections[key].clear()
--> 974         self.read_dict({key: value})
    975 
    976     def __delitem__(self, key):

/usr/lib/python3.9/configparser.py in read_dict(self, dictionary, source)
    745                     raise
    746             elements_added.add(section)
--> 747             for key, value in keys.items():
    748                 key = self.optionxform(str(key))
    749                 if value is not None:

AttributeError: 'NoneType' object has no attribute 'items'

[whyscream]

Important addition: if I include the specific ipdb snippet:

[tool.ipdb]
context = 10

to the pyproject.toml, it doesn't break anymore. So it seems your toml parsing is now correct, but only the extraction of the configuration value fails (especially when the value isn't in the config).

[alexandrebarbaruiva]

Thanks for the detailed answer! I think my latest changes have fixed that problem

[whyscream]

My limited testing shows that, after your latest changes, the traceback doesn't happen anymore. Thanks for the quick response!

[alexandrebarbaruiva]

I'll do some more testing to be sure

[gotcha]

@alexandrebarbaruiva Is #219 ready ?

[alexandrebarbaruiva]

Yes

[gotcha]

New https://pypi.org/project/ipdb/0.13.6/