search

kalekundert / byoc

MIT License
0 stars 0 forks source link

IOT instruction (core dumped) error in python3.8 #41

Closed kalekundert closed 2 years ago

kalekundert commented 2 years ago

The following tests generate IOT instruction (core dumped) errors with python 3.8, but not python 3.9 or 3.10:

Here's a console session:

$ pyenv shell 3.8.13
$ pip install -U --upgrade-strategy eager '.[test]'
$ pytest test_assembly.py -k test_parse_assemblies_from_docopt -v                                                                                (master !?$)
==================================================================== test session starts =====================================================================
platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0 -- /home/kale/.pyenv/versions/3.8.13/bin/python3.8
cachedir: .pytest_cache
rootdir: /home/kale/research/software/projects/stepwise_mol_bio, configfile: pyproject.toml
plugins: typeguard-2.13.3, cov-3.0.0, env-0.6.2
collected 26 items / 18 deselected / 8 selected

test_assembly.py::test_parse_assemblies_from_docopt[frags0] [1]    226557 IOT instruction (core dumped)  pytest test_assembly.py -k test_parse_assemblies_from_docopt -v

Getting rid of the Fragment.__repr__() function solves the problem. This makes me think that the issue is somehow related to the logging that BYOC is doing.

More specifically, the error occurs when the attrs variable in Fragment.__repr__() contains 'conc' and either 'mw' or 'length'. I expanded the list comprehension in that function into a for-loop, and that had no effect on the error. But when I added debug() calls between the various statements, the error went away. This makes me think that this is a bug in python, perhaps relating to catching a recursion error.

I can also prevent the error by commenting out the "getting {self._name!r} parameter for {format_obj(obj)}" log message in byoc.param._calc_value(). In fact, I can tell that the interpreter crashes just before entering the except Exception block in the format_obj() function within _calc_value(). This suggests that the bug has something to do with catching a recursion error, although I couldn't reproduce the bug with a simple MWE.

Overall, I'm starting to wonder if calling repr(obj) when I can't find a value for an attribute is just asking for trouble. It is nice to be able to distinguish objects of the same type, but I really don't want my logging code to be triggering these crazy-ass bugs. Even though I think this is probably a bug in python (that was fixed in 3.9 but not 3.8), logging code shouldn't be tempting fate.

kalekundert commented 2 years ago

I already dealt with this issue, but trying to make a MWE for the purpose of submitting a bug to python, I think I know what's going on a little more. Here's the MWE I came up with:

class descriptor:

    def __get__(self, obj, cls=None):
        try:
            obj.method()
        except RecursionError:
            pass

        return 'value'

class Obj:
    x = descriptor()
    y = descriptor()

    def method(self):
        return f'Obj({self.x!r}, {self.y!r})'

obj = Obj()
print(obj.x)

This isn't a perfect example, because in python>=3.8 it creates an infinite loop. I don't understand why, especially since my real code did not. In python==3.8, I get the following error message:

❯ py mwe.py
Fatal Python error: Cannot recover from stack overflow.
Python runtime state: initialized

Current thread 0x00007f0c9ea75740 (most recent call first):
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  File "mwe.py", line 7 in __get__
  File "mwe.py", line 18 in method
  ...
[1]    285536 IOT instruction (core dumped)  python3 mwe.py

So at least in this case, the problem is a stack overflow. My guess is that python fails to detect the recursion, and ultimately runs out of stack space.