search

nedbat / coveragepy

The code coverage tool for Python
https://coverage.readthedocs.io
Apache License 2.0
3.01k stars 432 forks source link

RecursionError encountered with python 3.10.4 and coverage 3.4.1 #1397

Closed vEpiphyte closed 2 years ago

vEpiphyte commented 2 years ago

Describe the bug When using Coverage on my project with Python 3.10.4, a RecursionError is encountered. This appears similar to https://github.com/nedbat/coveragepy/issues/1298 . The following shows executing a unit test with and without coverage, with coverage failing.

(covtest3104) epiphyte@vertex005c:/tmp/syn2/synapse$ python -m pytest synapse/tests/test_lib_provenance.py 
=============================================================== test session starts ================================================================
platform linux -- Python 3.10.4, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /tmp/syn2/synapse
plugins: forked-1.4.0, xdist-2.5.0, cov-3.0.0
collected 3 items                                                                                                                                  

synapse/tests/test_lib_provenance.py .ss                                                                                                     [100%]

=========================================================== 1 passed, 2 skipped in 3.05s ===========================================================

(covtest3104) epiphyte@vertex005c:/tmp/syn2/synapse$ python -m coverage run -m pytest synapse/tests/test_lib_provenance.py 
=============================================================== test session starts ================================================================
platform linux -- Python 3.10.4, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /tmp/syn2/synapse
plugins: forked-1.4.0, xdist-2.5.0, cov-3.0.0
collected 3 items                                                                                                                                  

synapse/tests/test_lib_provenance.py Fss                                                                                                     [100%]

===================================================================== FAILURES =====================================================================
_____________________________________________________________ ProvenanceTest.test_prov _____________________________________________________________

args = (), kwargs = {}, coro = <coroutine object ProvenanceTest.test_prov at 0x7fa1d82ab920>

    def wrap(*args, **kwargs):

        coro = f(*args, **kwargs)

        if not iAmLoop():
>           return sync(coro)

synapse/glob.py:118: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
synapse/glob.py:87: in sync
    return asyncio.run_coroutine_threadsafe(coro, loop).result(timeout)
/home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/concurrent/futures/_base.py:446: in result
    return self.__get_result()
/home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/concurrent/futures/_base.py:391: in __get_result
    raise self._exception
synapse/tests/test_lib_provenance.py:87: in test_prov
    await real.nodes(q)
synapse/cortex.py:4658: in nodes
    return await view.nodes(text, opts=opts)
synapse/lib/view.py:406: in nodes
    return [n async for n in self.eval(text, opts=opts)]
synapse/lib/view.py:406: in <listcomp>
    return [n async for n in self.eval(text, opts=opts)]
synapse/lib/view.py:374: in eval
    async for node in snap.eval(text, opts=opts, user=user):
synapse/lib/snap.py:578: in eval
    async for node, path in runt.execute():
synapse/lib/storm.py:1954: in execute
    async for item in self.query.iterNodePaths(self, genr=genr):
synapse/lib/ast.py:192: in iterNodePaths
    async for node, path in genr:
synapse/lib/ast.py:163: in run
    async for node, path in genr:
synapse/lib/ast.py:938: in run
    genr, empty = await pullone(genx())
synapse/lib/ast.py:892: in pullone
    async for gotone in genr:
synapse/lib/ast.py:928: in genx
    async for node, path in genr:
synapse/lib/ast.py:938: in run
    genr, empty = await pullone(genx())
E   RecursionError: maximum recursion depth exceeded in comparison
!!! Recursion detected (same locals & position)
============================================================= short test summary info ==============================================================
FAILED synapse/tests/test_lib_provenance.py::ProvenanceTest::test_prov - RecursionError: maximum recursion depth exceeded in comparison
=========================================================== 1 failed, 2 skipped in 5.11s ===========================================================

To Reproduce How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

  1. What version of Python are you using?

Python 3.10.4 (built using pyenv). This has also been seen in Python 3.10.4, available from CircleCI's cimg/python:3.10 docker container.

  1. What version of coverage.py shows the problem? The output of coverage debug sys is helpful.

    $ coverage debug sys
-- sys -------------------------------------------------------
           coverage_version: 6.4.1
            coverage_module: /home/epiphyte/.pyenv/versions/covtest3104/lib/python3.10/site-packages/coverage/__init__.py
                     tracer: -none-
                    CTracer: available
       plugins.file_tracers: -none-
        plugins.configurers: -none-
  plugins.context_switchers: -none-
          configs_attempted: .coveragerc
               configs_read: /tmp/syn2/synapse/.coveragerc
                config_file: /tmp/syn2/synapse/.coveragerc
            config_contents: b'[report]\nomit =\n    */synapse/tests/test_*\n'
                  data_file: -none-
                     python: 3.10.4 (main, Apr 26 2022, 09:52:13) [GCC 11.2.0]
                   platform: Linux-5.13.0-46-generic-x86_64-with-glibc2.34
             implementation: CPython
                 executable: /home/epiphyte/.pyenv/versions/covtest3104/bin/python
               def_encoding: utf-8
                fs_encoding: utf-8
                        pid: 609634
                        cwd: /tmp/syn2/synapse
                       path: /home/epiphyte/.pyenv/versions/3.10.4/envs/covtest3104/bin
                             /home/epiphyte/.pyenv/versions/3.10.4/lib/python310.zip
                             /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10
                             /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/lib-dynload
                             /home/epiphyte/.pyenv/versions/covtest3104/lib/python3.10/site-packages
                environment: HOME = /home/epiphyte
                             PYENV_DIR = /tmp/syn2/synapse
                             PYENV_HOOK_PATH = /home/epiphyte/.pyenv/pyenv.d:/usr/local/etc/pyenv.d:/etc/pyenv.d:/usr/lib/pyenv/hooks:/home/epiphyte/.pyenv/plugins/pyenv-virtualenv/etc/pyenv.d:/home/epiphyte/.pyenv/plugins/pyenv-which-ext/etc/pyenv.d
                             PYENV_ROOT = /home/epiphyte/.pyenv
                             PYENV_VERSION = covtest3104
                             PYENV_VIRTUALENV_INIT = 1
                             PYENV_VIRTUAL_ENV = /home/epiphyte/.pyenv/versions/3.10.4/envs/covtest3104
                             PYTHONSTARTUP = /home/epiphyte/.pythonrc
                             PYTHON_CONFIGURE_OPTS = --enable-shared
               command_line: /home/epiphyte/.pyenv/versions/covtest3104/bin/coverage debug sys
            sqlite3_version: 2.6.0
     sqlite3_sqlite_version: 3.35.5
         sqlite3_temp_store: 0
    sqlite3_compile_options: COMPILER=gcc-11.2.0, ENABLE_COLUMN_METADATA, ENABLE_DBSTAT_VTAB,
                             ENABLE_FTS3, ENABLE_FTS3_PARENTHESIS, ENABLE_FTS3_TOKENIZER, ENABLE_FTS4,
                             ENABLE_FTS5, ENABLE_JSON1, ENABLE_LOAD_EXTENSION, ENABLE_MATH_FUNCTIONS,
                             ENABLE_PREUPDATE_HOOK, ENABLE_RTREE, ENABLE_SESSION, ENABLE_STMTVTAB,
                             ENABLE_UNLOCK_NOTIFY, ENABLE_UPDATE_DELETE_LIMIT, HAVE_ISNAN,
                             LIKE_DOESNT_MATCH_BLOBS, MAX_SCHEMA_RETRY=25, MAX_VARIABLE_NUMBER=250000,
                             OMIT_LOOKASIDE, SECURE_DELETE, SOUNDEX, TEMP_STORE=1, THREADSAFE=1, USE_URI

  2. What versions of what packages do you have installed? The output of pip freeze is helpful.

$ python -m pip freeze
aiohttp==3.8.1
aiohttp-socks==0.6.1
aioimaplib==0.9.0
aiosignal==1.2.0
aiosmtplib==1.1.6
antlr4-python3-runtime==4.9.3
appdirs==1.4.4
async-timeout==4.0.2
attrs==21.4.0
autopep8==1.6.0
base58==2.1.1
bech32==1.2.0
bump2version==1.0.1
cattrs==1.10.0
cbor2==5.4.2.post1
certifi==2022.5.18.1
cffi==1.15.0
charset-normalizer==2.0.12
colorama==0.4.4
coverage==6.4.1
cpe==1.2.1
cryptography==37.0.2
execnet==1.9.0
fastjsonschema==2.15.3
frozenlist==1.3.0
idna==3.3
iniconfig==1.1.1
jsonschema==4.6.0
lark==1.1.2
lmdb==1.3.0
more-itertools==8.13.0
msgpack==1.0.4
multidict==6.0.2
oauthlib==3.2.0
packaging==21.3
pluggy==1.0.0
prompt-toolkit==3.0.29
py==1.11.0
pycodestyle==2.8.0
pycparser==2.21
pycryptodome==3.14.1
Pygments==2.11.2
pyOpenSSL==22.0.0
pyparsing==3.0.9
pyrsistent==0.18.1
pytest==6.2.5
pytest-cov==3.0.0
pytest-forked==1.4.0
pytest-xdist==2.5.0
python-bitcoinlib==0.11.0
python-dateutil==2.8.2
python-socks==1.2.4
pytz==2022.1
PyYAML==6.0
regex==2022.6.2
requests==2.27.1
requests-cache==0.9.4
scalecodec==1.0.34
simplejson==3.17.6
six==1.16.0
stix2-patterns==2.0.0
stix2-validator==3.0.2
toml==0.10.2
tomli==2.0.1
tornado==6.1
typing_extensions==4.2.0
url-normalize==1.4.3
urllib3==1.26.9
vcrpy==4.1.1
wcwidth==0.2.5
wrapt==1.14.1
xxhash==3.0.0
yarl==1.7.2
  1. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.

I don't have a small reproduction - this is occurring inside of test code that is triggering application level recursion safety checks. This is public code so it should be reproducible.

git clone https://github.com/vertexproject/synapse.git
# Setup a python 3.10.4 venv, checkout the branch and install deps
cd synapse
git checkout bug_provtest_coverage
python -m pip install -r requirements_dev.txt
# Then run the test from earlier - the first should work
python -m pytest synapse/tests/test_lib_provenance.py  
# The test with coverage should fail
python -m coverage run -m pytest synapse/tests/test_lib_provenance.py

Expected behavior Do not encounter Python RecursionErrors if possible :D

Additional context I'm more than happy to try various coverage patches or assist with any additional information needed to help understand this issue.

vEpiphyte commented 2 years ago

Something extra weird - using breakpoint() to step to the location of the recursion error gives me a different RecursionError. This is available on the bug_provtext_coverage_bp branch of the synapse project.

(syn3104) epiphyte@vertex005c:~/PycharmProjects/synapse$ git reflog -n 1
6b6eeb81 (HEAD -> bug_provtest_coverage_bp, origin/bug_provtest_coverage_bp) HEAD@{0}: commit: Add breakpoint
(syn3104) epiphyte@vertex005c:~/PycharmProjects/synapse$ git rev-parse HEAD
6b6eeb81a39a4c47e4cfdebf4004a455250ce36e
(syn3104) epiphyte@vertex005c:~/PycharmProjects/synapse$ python -m coverage run -m pytest -v -s -rs synapse/tests/test_lib_provenance.py 
========================================================================================= test session starts =========================================================================================
platform linux -- Python 3.10.4, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- /home/epiphyte/.pyenv/versions/3.10.4/envs/syn3104/bin/python
cachedir: .pytest_cache
rootdir: /home/epiphyte/PycharmProjects/synapse, configfile: pytest.ini
plugins: forked-1.4.0, xdist-2.5.0, cov-3.0.0
collecting ... callling colorama init
post colorama init
collected 3 items                                                                                                                                                                                     

synapse/tests/test_lib_provenance.py::ProvenanceTest::test_prov 
------------------------------------------------------------ live log call -----------------------------------------------------
2022-06-09 08:49:27 [DEBUG] Using selector: EpollSelector [selector_events.py:__init__:MainThread:MainProcess]

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(84)claim()
-> if stacklen > 256:
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(90)claim()
-> if not ProvenanceEnabled:
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(93)claim()
-> stack.push(typ, **info)
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(57)push()
-> def push(self, typ, **info):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(58)push()
-> tuplinfo = tuple((k, info[k]) for k in sorted(info.keys()))
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(58)<genexpr>()
-> tuplinfo = tuple((k, info[k]) for k in sorted(info.keys()))
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(58)<genexpr>()
-> tuplinfo = tuple((k, info[k]) for k in sorted(info.keys()))
(Pdb) s
--Return--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(58)<genexpr>()->('name', 'uniq')
-> tuplinfo = tuple((k, info[k]) for k in sorted(info.keys()))
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(58)<genexpr>()->('name', 'uniq')
-> tuplinfo = tuple((k, info[k]) for k in sorted(info.keys()))
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(58)<genexpr>()->('name', 'uniq')
-> tuplinfo = tuple((k, info[k]) for k in sorted(info.keys()))
(Pdb) s
--Return--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(58)<genexpr>()->None
-> tuplinfo = tuple((k, info[k]) for k in sorted(info.keys()))
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(59)push()
-> self.provs.append((typ, tuplinfo))
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(60)push()
-> self.idens.append((None, True))
(Pdb) s
--Return--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(60)push()->None
-> self.idens.append((None, True))
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(95)claim()
-> try:
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(96)claim()
-> yield
(Pdb) s
--Return--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/provenance.py(96)claim()->None
-> yield
(Pdb) s
--Return--
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/contextlib.py(135)__enter__()->None
-> return next(self.gen)
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(926)run()
-> async def genx():
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(938)run()
-> genr, empty = await pullone(genx())
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(890)pullone()
-> async def pullone(genr):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(891)pullone()
-> gotone = None
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(892)pullone()
-> async for gotone in genr:
(Pdb) s
--Call--
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py(526)_asyncgen_firstiter_hook()
-> def _asyncgen_firstiter_hook(self, agen):
(Pdb) s
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py(527)_asyncgen_firstiter_hook()
-> if self._asyncgens_shutdown_called:
(Pdb) s
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py(533)_asyncgen_firstiter_hook()
-> self._asyncgens.add(agen)
(Pdb) s
--Call--
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/_weakrefset.py(86)add()
-> def add(self, item):
(Pdb) s
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/_weakrefset.py(87)add()
-> if self._pending_removals:
(Pdb) s
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/_weakrefset.py(89)add()
-> self.data.add(ref(item, self._remove))
(Pdb) s
--Return--
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/_weakrefset.py(89)add()->None
-> self.data.add(ref(item, self._remove))
(Pdb) s
--Return--
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py(533)_asyncgen_firstiter_hook()->None
-> self._asyncgens.add(agen)
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(926)genx()
-> async def genx():
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(928)genx()
-> async for node, path in genr:
(Pdb) s
--Call--
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py(526)_asyncgen_firstiter_hook()
-> def _asyncgen_firstiter_hook(self, agen):
(Pdb) s
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py(527)_asyncgen_firstiter_hook()
-> if self._asyncgens_shutdown_called:
(Pdb) s
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py(533)_asyncgen_firstiter_hook()
-> self._asyncgens.add(agen)
(Pdb) s
--Call--
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/_weakrefset.py(86)add()
-> def add(self, item):
(Pdb) s
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/_weakrefset.py(87)add()
-> if self._pending_removals:
(Pdb) s
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/_weakrefset.py(89)add()
-> self.data.add(ref(item, self._remove))
(Pdb) s
--Return--
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/_weakrefset.py(89)add()->None
-> self.data.add(ref(item, self._remove))
(Pdb) s
--Return--
> /home/epiphyte/.pyenv/versions/3.10.4/lib/python3.10/asyncio/base_events.py(533)_asyncgen_firstiter_hook()->None
-> self._asyncgens.add(agen)
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(908)run()
-> async def run(self, runt, genr):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(910)run()
-> name = await self.kids[0].compute(runt, None)
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(2978)compute()
-> async def compute(self, runt, path):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(2979)compute()
-> return self.valu
(Pdb) s
--Return--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(2979)compute()->'uniq'
-> return self.valu
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(912)run()
-> ctor = runt.snap.core.getStormCmd(name)
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/cortex.py(4269)getStormCmd()
-> def getStormCmd(self, name):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/cortex.py(4270)getStormCmd()
-> return self.stormcmds.get(name)
(Pdb) s
--Return--
> /home/epiphyte/PycharmProjects/synapse/synapse/cortex.py(4270)getStormCmd()-><class 'synap...torm.UniqCmd'>
-> return self.stormcmds.get(name)
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(913)run()
-> if ctor is None:
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(917)run()
-> runtsafe = self.kids[1].isRuntSafe(runt)
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(2972)isRuntSafe()
-> def isRuntSafe(self, runt):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(2973)isRuntSafe()
-> return True
(Pdb) s
--Return--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(2973)isRuntSafe()->True
-> return True
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/ast.py(919)run()
-> scmd = ctor(runt, runtsafe)
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2438)__init__()
-> def __init__(self, runt, runtsafe):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2440)__init__()
-> self.opts = None
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2441)__init__()
-> self.argv = None
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2443)__init__()
-> self.runt = runt
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2444)__init__()
-> self.runtsafe = runtsafe
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2446)__init__()
-> self.pars = self.getArgParser()
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(3223)getArgParser()
-> def getArgParser(self):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(3224)getArgParser()
-> pars = Cmd.getArgParser(self)
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2462)getArgParser()
-> def getArgParser(self):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2463)getArgParser()
-> return Parser(prog=self.getName(), descr=self.getDescr())
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2456)getName()
-> def getName(self):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2457)getName()
-> return self.name
(Pdb) s
--Return--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2457)getName()->'uniq'
-> return self.name
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2459)getDescr()
-> def getDescr(self):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2460)getDescr()
-> return self.__class__.__doc__
(Pdb) s
--Return--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2460)getDescr()->'\n    Filter... uniq\n\n    '
-> return self.__class__.__doc__
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2064)__init__()
-> def __init__(self, prog=None, descr=None, root=None):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2066)__init__()
-> if root is None:
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2067)__init__()
-> root = self
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2069)__init__()
-> self.prog = prog
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2070)__init__()
-> self.descr = descr
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2072)__init__()
-> self.root = root
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2073)__init__()
-> self.exited = False
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2074)__init__()
-> self.mesgs = []
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2076)__init__()
-> self.optargs = {}
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2077)__init__()
-> self.posargs = []
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2078)__init__()
-> self.allargs = []
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2080)__init__()
-> self.inputs = None
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2082)__init__()
-> self.reqopts = []
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2084)__init__()
-> self.add_argument('--help', '-h', action='store_true', default=False, help='Display the command usage.')
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2089)add_argument()
-> def add_argument(self, *names, **opts):
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2091)add_argument()
-> assert len(names)
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2093)add_argument()
-> argtype = opts.get('type')
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2094)add_argument()
-> if argtype is not None and argtype not in s_datamodel.Model().types:
(Pdb) s
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2099)add_argument()
-> dest = self._get_dest(names)
(Pdb) s
--Call--
> /home/epiphyte/PycharmProjects/synapse/synapse/lib/storm.py(2110)_get_dest()
-> def _get_dest(self, names):FAILED
synapse/tests/test_lib_provenance.py::ProvenanceTest::test_prov_disabled SKIPPED (test)
synapse/tests/test_lib_provenance.py::ProvenanceTest::test_prov_no_extra SKIPPED (Pending provenance cache)

===========================================FAILURES ==================================
___________________________ProvenanceTest.test_prov _____________________________

args = (), kwargs = {}, coro = <coroutine object ProvenanceTest.test_prov at 0x7f9ee8604e40>

    def wrap(*args, **kwargs):

        coro = f(*args, **kwargs)

        if not iAmLoop():
>           return sync(coro)

synapse/glob.py:118: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
synapse/glob.py:87: in sync
    return asyncio.run_coroutine_threadsafe(coro, loop).result(timeout)
../../.pyenv/versions/3.10.4/lib/python3.10/concurrent/futures/_base.py:446: in result
    return self.__get_result()
../../.pyenv/versions/3.10.4/lib/python3.10/concurrent/futures/_base.py:391: in __get_result
    raise self._exception
synapse/tests/test_lib_provenance.py:87: in test_prov
    await real.nodes(q)
synapse/cortex.py:4658: in nodes
    return await view.nodes(text, opts=opts)
synapse/lib/view.py:406: in nodes
    return [n async for n in self.eval(text, opts=opts)]
synapse/lib/view.py:406: in <listcomp>
    return [n async for n in self.eval(text, opts=opts)]
synapse/lib/view.py:374: in eval
    async for node in snap.eval(text, opts=opts, user=user):
synapse/lib/snap.py:578: in eval
    async for node, path in runt.execute():
synapse/lib/storm.py:1954: in execute
    async for item in self.query.iterNodePaths(self, genr=genr):
synapse/lib/ast.py:192: in iterNodePaths
    async for node, path in genr:
synapse/lib/ast.py:163: in run
    async for node, path in genr:
synapse/lib/ast.py:938: in run
    genr, empty = await pullone(genx())
synapse/lib/ast.py:892: in pullone
    async for gotone in genr:
synapse/lib/ast.py:928: in genx
    async for node, path in genr:
synapse/lib/ast.py:938: in run
    genr, empty = await pullone(genx())
E   RecursionError: maximum recursion depth exceeded while calling a Python object
!!! Recursion detected (same locals & position)
nedbat commented 2 years ago

The instructions from the first comment reproduce the problem for me. I don't know what's going on yet.

nedbat commented 2 years ago

I think coverage is not causing the recursion error, but is making it more likely. If I add sys.setrecursionlimit(775) to the top of test_lib_provenance.py, then it fails with the same error even without coverage measurement. I don't see what the change is in the stack when running coverage, but it seems to be adding enough frames somewhere to make the deep recursion hit the limit sooner.

I doubt there's anything I can change in coverage to make this different.