OSError: could not get source code when trying to use @guidance

[this-josh]

The bug Cannot using the @guidance decorator in a clean python 3.12 environment

To Reproduce

(.venv)➜  w pip freeze | grep guidance                                                                                             [1/May/24 | 15:22]
guidance==0.1.13
(.venv)➜  w python                                                                                                                 [1/May/24 | 15:22]
Python 3.12.3 | packaged by conda-forge | (main, Apr 15 2024, 18:35:20) [Clang 16.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import guidance
>>> import guidance
>>>
>>> @guidance
... def one_line_thing(lm, thing, topic):
...     lm += f'Here is a one-line {thing} about {topic}: ' + gen(stop='\n')
...     return lm # return our updated model
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/tmp/w/.venv/lib/python3.12/site-packages/guidance/__init__.py", line 19, in __call__
    return _decorator(f, stateless=stateless, cache=cache, dedent=dedent, model=model)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/w/.venv/lib/python3.12/site-packages/guidance/__init__.py", line 35, in _decorator
    f = strip_multiline_string_indents(f)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/w/.venv/lib/python3.12/site-packages/guidance/_utils.py", line 75, in strip_multiline_string_indents
    source = textwrap.dedent(inspect.getsource(f))
                             ^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/w/.venv/lib/python3.12/inspect.py", line 1278, in getsource
    lines, lnum = getsourcelines(object)
                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/w/.venv/lib/python3.12/inspect.py", line 1260, in getsourcelines
    lines, lnum = findsource(object)
                  ^^^^^^^^^^^^^^^^^^
  File "/private/tmp/w/.venv/lib/python3.12/inspect.py", line 1089, in findsource
    raise OSError('could not get source code')
OSError: could not get source code
>>>

System info (please complete the following information):

• OS (e.g. Ubuntu, Windows 11, Mac OS, etc.): Mac OS
• Guidance Version (guidance.__version__):0.1.13

[Harsha-Nori]

I think this is caused by having a PyPI package install that somehow mismatches your Python version. Can you try uninstalling guidance and doing a clean re-install with no-cache dirs?

If you installed via pip:

pip uninstall guidance && pip install guidance --no-cache-dir

might do the trick. I don't think this error is specific to guidance though, I can't reproduce it on my mac

[this-josh]

Thanks this has fixed it. I got the same error on my Mac and Linux based HPC cluster so forgot about both having the same local cache issue.

[this-josh]

@Harsha-Nori Sorry the issue seems to be reoccurring now, again in a new environment

(.venv)➜  guidance pip install guidance --no-cache-dir                                                                                 [2/May/24 | 14:18]
(.venv)➜  guidance python                                                                                                              [2/May/24 | 14:18]
Python 3.12.3 | packaged by conda-forge | (main, Apr 15 2024, 18:35:20) [Clang 16.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import guidance
>>> from guidance import  select, gen
>>> from guidance import models
>>>
>>> @guidance
... def character_maker(lm, id, description, valid_weapons):
...     lm += f"""\
...     The following is a character profile for an RPG game in JSON format.
...     ```json
...     {{
...         "id": "{id}",
...         "description": "{description}",
...         "name": "{gen('name', stop='"')}",
...         "age": {gen('age', regex='[0-9]+', stop=',')},
...         "armor": "{select(options=['leather', 'chainmail', 'plate'], name='armor')}",
...         "weapon": "{select(options=valid_weapons, name='weapon')}",
...         "class": "{gen('class', stop='"')}",
...         "mantra": "{gen('mantra', stop='"')}",
...         "strength": {gen('strength', regex='[0-9]+', stop=',')},
...         "items": ["{gen('item', list_append=True, stop='"')}", "{gen('item', list_append=True, stop='"')}", "{gen('item', list_append=True, stop='"')}"]
...     }}```"""
...     return lm
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/tmp/guidance/.venv/lib/python3.12/site-packages/guidance/__init__.py", line 19, in __call__
    return _decorator(f, stateless=stateless, cache=cache, dedent=dedent, model=model)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/guidance/.venv/lib/python3.12/site-packages/guidance/__init__.py", line 35, in _decorator
    f = strip_multiline_string_indents(f)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/guidance/.venv/lib/python3.12/site-packages/guidance/_utils.py", line 75, in strip_multiline_string_indents
    source = textwrap.dedent(inspect.getsource(f))
                             ^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/guidance/.venv/lib/python3.12/inspect.py", line 1278, in getsource
    lines, lnum = getsourcelines(object)
                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/guidance/.venv/lib/python3.12/inspect.py", line 1260, in getsourcelines
    lines, lnum = findsource(object)
                  ^^^^^^^^^^^^^^^^^^
  File "/private/tmp/guidance/.venv/lib/python3.12/inspect.py", line 1089, in findsource
    raise OSError('could not get source code')
OSError: could not get source code

[Harsha-Nori]

Hi @this-josh, I'm having an impossible time reproducing this on my mac in fresh environments :(. If you want to avoid this codepath, you can simply set @guidance(dedent=False) in the decorator. This just means you'll have to be a bit more careful about multi-line whitespaces in f-strings (see dedent info here: https://docs.python.org/3/library/textwrap.html#textwrap.dedent), but it won't change anything about guidance's functionality.

[this-josh]

Thanks @Harsha-Nori for looking into it. That is unfortunate as I get the same error on my mac and an entirely clean Linux HPC instance using both Python 3.11 and 3.12.

Perhaps someone else could try reproducing it?

[this-josh]

Here is a complete gist of the STDOUT

[this-josh]

The issue seems to be running from an interactive shell

(.venv)➜  guidance bat run.py                                                                                                     [13/May/24 | 10:43]
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: run.py
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ import guidance
   2   │ from guidance import  select, gen
   3   │ from guidance import models
   4   │
   5   │
   6   │ @guidance
   7   │ def character_maker(lm, id, description, valid_weapons):
   8   │     lm += f"""\
   9   │     The following is a character profile for an RPG game in JSON format.
  10   │     ```json
  11   │     {{
  12   │         "id": "{id}",
  13   │         "description": "{description}",
  14   │         "name": "{gen('name', stop='"')}",
  15   │         "age": {gen('age', regex='[0-9]+', stop=',')},
  16   │         "armor": "{select(options=['leather', 'chainmail', 'plate'], name='armor')}",
  17   │         "weapon": "{select(options=valid_weapons, name='weapon')}",
  18   │         "class": "{gen('class', stop='"')}",
  19   │         "mantra": "{gen('mantra', stop='"')}",
  20   │         "strength": {gen('strength', regex='[0-9]+', stop=',')},
  21   │         "items": ["{gen('item', list_append=True, stop='"')}", "{gen('item', list_append=True, stop='"')}", "{gen('item', list_append=True, s
       │ top='"')}"]
  22   │     }}```"""
  23   │     return lm
  24   │
  25   │
  26   │ print('done!')
───────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
(.venv)➜  guidance python run.py    # this runs fine                                                                                              [13/May/24 | 10:43]
done!
(.venv)➜  guidance python                                                                                                         [13/May/24 | 10:43]
Python 3.12.3 | packaged by conda-forge | (main, Apr 15 2024, 18:35:20) [Clang 16.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import guidance
from guidance import  select, gen^Mfrom guidance import models^M^M^M@guidance^Mdef character_maker(lm, id, description, valid_weapons):^M    lm += f"""\^M    The following is a character profile for an RPG game in JSON format.^M    ```json^M    {{^M        "id": "{id}",^M        "description": "{description}",^M        "name": "{gen('name', stop='"')}",^M        "age": {gen('age', regex='[0-9]+', stop=',')},^M        "armor": "{select(options=['leather', 'chainmail', 'plate'], name='armor')}",^M        "weapon": "{select(options=valid_weapons, name='weapon')}",^M        "class": "{gen('class', stop='"')}",^M        "mantra": "{gen('mantra', stop='"')}",^M        "strength": {gen('strength', regex='[0-9]+', stop=',')},^M        "items": ["{gen('item', list_append=True, stop='"')}", "{gen('item', list_append=True, stop='"')}", "{gen('item', list_append=True, stop='"')}"]^M    }}```"""^M    return lm^M^M>>> from guidance import  select, gen
>>> from guidance import models
>>>
>>>
>>> @guidance
... def character_maker(lm, id, description, valid_weapons):
...     lm += f"""\
...     The following is a character profile for an RPG game in JSON format.
...     ```json
...     {{
...         "id": "{id}",
...         "description": "{description}",
...         "name": "{gen('name', stop='"')}",
...         "age": {gen('age', regex='[0-9]+', stop=',')},
...         "armor": "{select(options=['leather', 'chainmail', 'plate'], name='armor')}",
...         "weapon": "{select(options=valid_weapons, name='weapon')}",
...         "class": "{gen('class', stop='"')}",
...         "mantra": "{gen('mantra', stop='"')}",
...         "strength": {gen('strength', regex='[0-9]+', stop=',')},
...         "items": ["{gen('item', list_append=True, stop='"')}", "{gen('item', list_append=True, stop='"')}", "{gen('item', list_append=True, stop='"')}"]
...     }}```"""
...     return lm
...

OSError: could not get source code
>>>

[mmoskal]

I had an issue with stale __pycache__ folders. Removing them helped.