Tracing within an exec'd string does not work

skeledrew commented 3 years ago

I am attempting to use birdseye from within another tool, and a part of what it does is modify the source dynamically. This causes attempts to get a filename to fail. Is there a way to get around this without, say creating a temporary file?

...
  File "/home/skeledrew/Projects/pyls-livepy/.nox/make_pyls_livepy_test_env/lib/python3.7/site-packages/pyls_livepy/tracer.py", line 108, in trace_and_report
    report = run_trace_2(modded_source)
             │           └ 'import pyls_livepy\n\n@pyls_livepy.eye\ndef func(x):\n    """\n    >>> func(5)\n    0\n    """\n    return x * x\n\nfunc(5)\n'
             └ <function run_trace_2 at 0x7fe36615d830>
> File "/home/skeledrew/Projects/pyls-livepy/.nox/make_pyls_livepy_test_env/lib/python3.7/site-packages/pyls_livepy/tracer.py", line 78, in run_trace_2
    exec(source)
         └ 'import pyls_livepy\n\n@pyls_livepy.eye\ndef func(x):\n    """\n    >>> func(5)\n    0\n    """\n    return x * x\n\nfunc(5)\n'
  File "<string>", line 3, in <module>
  File "/home/skeledrew/Projects/pyls-livepy/.nox/make_pyls_livepy_test_env/lib/python3.7/site-packages/birdseye/__init__.py", line 24, in __call__
    return self.__val()(*args, **kwargs)
           │             │       └ {}
           │             └ (<function func at 0x7fe369310e60>,)
           └ <birdseye._SimpleProxy object at 0x7fe368e31750>
  File "/home/skeledrew/Projects/pyls-livepy/.nox/make_pyls_livepy_test_env/lib/python3.7/site-packages/birdseye/tracer.py", line 273, in __call__
    return self.trace_function(func)
           │    │              └ <function func at 0x7fe369310e60>
           │    └ <function BirdsEye.trace_function at 0x7fe3671f07a0>
           └ <birdseye.bird.BirdsEye object at 0x7fe3671ef110>
  File "/home/skeledrew/Projects/pyls-livepy/.nox/make_pyls_livepy_test_env/lib/python3.7/site-packages/birdseye/bird.py", line 410, in trace_function
    new_func = super(BirdsEye, self).trace_function(func)
                     │         │                    └ <function func at 0x7fe369310e60>
                     │         └ <birdseye.bird.BirdsEye object at 0x7fe3671ef110>
                     └ <class 'birdseye.bird.BirdsEye'>
  File "/home/skeledrew/Projects/pyls-livepy/.nox/make_pyls_livepy_test_env/lib/python3.7/site-packages/birdseye/tracer.py", line 204, in trace_function
    source = read_source_file(filename)
             │                └ '<string>'
             └ <function read_source_file at 0x7fe36841d8c0>
  File "/home/skeledrew/Projects/pyls-livepy/.nox/make_pyls_livepy_test_env/lib/python3.7/site-packages/birdseye/utils.py", line 202, in read_source_file
    with open_with_encoding_check(filename) as f:
         │                        └ '<string>'
         └ <function open at 0x7fe369e3c4d0>
  File "/home/skeledrew/.conda/envs/utils_env/lib/python3.7/tokenize.py", line 447, in open
    buffer = _builtin_open(filename, 'rb')
             │             └ '<string>'
             └ <built-in function open>

FileNotFoundError: [Errno 2] No such file or directory: '<string>'

alexmojaki commented 3 years ago

If you're willing to make a PR to make read_source_file use linecache.getlines instead of the filesystem, then you could add a linecache entry like this:

https://github.com/gristlabs/grist-core/blob/8524b4f791dceea4d36e3fccec1455d3195ea629/sandbox/grist/gencode.py#L201-L210

The end result would be basically the same as creating a file, but without involving actual files.

Whether you do this or make a temporary file, there will also be the benefit of the source code showing up in a traceback, which as you can see it currently doesn't.

I've complained about this problem in general here but got no response.

skeledrew commented 3 years ago

I made a PR here. Seems a bit hacky though, as it'll require anyone else who runs into the issue to be aware of the implementation.

alexmojaki commented 3 years ago

Great! Can you confirm that using that branch and the modified form of exec fixes the problem on your end? Are tracebacks also working now?

skeledrew commented 3 years ago

Working great, thanks! Though I didn't really notice an issue with the traceback TBH.

alexmojaki commented 3 years ago

Look at what you originally posted:

> File "/home/skeledrew/Projects/pyls-livepy/.nox/make_pyls_livepy_test_env/lib/python3.7/site-packages/pyls_livepy/tracer.py", line 78, in run_trace_2
    exec(source)
         └ 'import pyls_livepy\n\n@pyls_livepy.eye\ndef func(x):\n    """\n    >>> func(5)\n    0\n    """\n    return x * x\n\nfunc(5)\n'
  File "<string>", line 3, in <module>
  File "/home/skeledrew/Projects/pyls-livepy/.nox/make_pyls_livepy_test_env/lib/python3.7/site-packages/birdseye/__init__.py", line 24, in __call__
    return self.__val()(*args, **kwargs)

There's no source code under File "<string>", line 3, in <module>

skeledrew commented 3 years ago

Oh I see. I thought it was just a normal limitation. Yes I do have full tracebacks now, thanks.

> File "/home/skeledrew/Projects/pyls-livepy/.nox/make_pyls_livepy_test_env/lib/python3.7/site-packages/pyls_livepy/tracer.py", line 58, in run_trace
    exec(code_obj)
         └ <code object <module> at 0x7f4dd830b6f0, file "<some-string>", line 1>
  File "<some-string>", line 32, in <module>
    square("5")
    └ <function square at 0x7f4dd81d4320>
  File "<some-string>", line 20, in square
    return x ** 2
           └ '5'

alexmojaki / birdseye

Tracing within an exec'd string does not work #94