Reproducible NameError with recursive function in PythonREPL()

System Info

MacOS, Python 3.11.2, langchain 0.0.174

Who can help?

N/A

Information

[ ] The official example notebooks/scripts
[X] My own modified scripts

Related Components

[ ] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[X] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction

I hit a strange error when using PythonREPL in an agent. It happens with this recursive Python function. It was written by gpt-3.5-turbo and is valid Python:

def fibonacci(n):
    if n <= 1:
        return n
    else:
        return fibonacci(n-1) + fibonacci(n-2)  # <-- this line will trigger the NameError
print(fibonacci(10))

Note that everything works as expected for non-recursive functions.

Using this function as string input, I reduced the issue to this minimal example:

from langchain.utilities import PythonREPL

python_repl = PythonREPL()
cmd = "def fibonacci(n):\n    if n <= 1:\n        return n\n    else:\n        return fibonacci(n-1) + fibonacci(n-2)\nprint(fibonacci(10))"
python_repl.run(cmd)

$ python t.py NameError("name 'fibonacci' is not defined")

When executed with exec(cmd), it runs as expected. I found that PythonREPL runs the command with exec(command, self.globals, self.locals).

Setting globals & locals in this python_repl instance makes the fragment work as expected:

# [... as before ...]
python_repl.globals = globals()
python_repl.locals = locals()
python_repl.run(cmd)

This hack solves it only in the context of this simple example, but not if python_repl added as a tool to an AgentExecutor.

At the core, the issue seems to be caused by Python scopes, but my knowledge of this is not profound enough to fully understand what's happening.

Expected behavior

I would have expected PythonREPL to accept recursive functions.

Relevant Files

langchain/utilities/python.py

Explanation

langchain/utilities/python.py

To fix the issue with the recursive function in PythonREPL, we need to update the run method to ensure that the function is defined in the correct scope. We can do this by updating the exec call to include the self.locals dictionary in the self.globals dictionary before executing the command. This will ensure that the function is defined in the correct scope and can be called recursively.

Here's the updated run method:

def run(self, command: str) -> str:
    """Run command with own globals/locals and returns anything printed."""
    old_stdout = sys.stdout
    sys.stdout = mystdout = StringIO()

    # Update the globals dictionary with the locals dictionary
    self.globals.update(self.locals)

    try:
        exec(command, self.globals, self.locals)
        sys.stdout = old_stdout
        output = mystdout.getvalue()
    except Exception as e:
        sys.stdout = old_stdout
        output = repr(e)
    return output

With this change, the PythonREPL should now work correctly with recursive functions.

devstein / langchain