Unable to save playwright Page object in the Sqlite database of langraph

Zubair2019 commented 5 days ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangGraph/LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangGraph/LangChain rather than my code.
[X] I am sure this is better as an issue rather than a GitHub discussion, since this is a LangGraph bug and not a design question.

Example Code

class AgentState(TypedDict):
    task: str
    browser_page: Optional[Page]

def run(playwright: Playwright) -> None:
        browser =  playwright.chromium.launch(headless=False)
        context =  browser.new_context()
        page =  context.new_page()
        with SqliteSaver.from_conn_string(":memory:") as memory:
            graph = builder.compile(checkpointer=memory)
            thread = {"configurable": {"thread_id": "1"}}
            try:
                for s in graph.stream({
                    'task': "",
                    "max_revisions": 0,
                    "revision_number": 1,
                    "browser_page":page
                }, thread):
                    print(s)
            except Exception as e:
                print(f"Error in astream: {e}")
                raise e

Error Message and Stack Trace (if applicable)

Error in astream: Object of type Page is not serializable
Traceback (most recent call last):
File "/Users/air/Github/0dfx/plan_agent/new.py", line 201, in <module>
run(playwright)
File "/Users/air/Github/0dfx/plan_agent/new.py", line 196, in run
raise e
File "/Users/air/Github/0dfx/plan_agent/new.py", line 186, in run
for s in graph.stream({
^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/pregel/init.py", line 1240, in stream
with SyncPregelLoop(
^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/pregel/loop.py", line 754, in exit
return self.stack.exit(exc_type, exc_value, traceback)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/lang/lib/python3.12/contextlib.py", line 610, in exit
raise exc_details[1]
File "/opt/anaconda3/envs/lang/lib/python3.12/contextlib.py", line 595, in exit
if cb(*exc_details):
^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/pregel/executor.py", line 105, in exit
task.result()
File "/opt/anaconda3/envs/lang/lib/python3.12/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/lang/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/pregel/loop.py", line 680, in _checkpointer_put_after_previous
prev.result()
File "/opt/anaconda3/envs/lang/lib/python3.12/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/lang/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/pregel/executor.py", line 70, in done
task.result()
File "/opt/anaconda3/envs/lang/lib/python3.12/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/lang/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/opt/anaconda3/envs/lang/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/pregel/loop.py", line 682, in checkpointer_put_after_previous
cast(BaseCheckpointSaver, self.checkpointer).put(
File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/checkpoint/sqlite/init.py", line 399, in put
type, serialized_checkpoint = self.serde.dumps_typed(checkpoint)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/checkpoint/serde/jsonplus.py", line 192, in dumps_typed
return "msgpack", _msgpack_enc(obj)
^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/checkpoint/serde/jsonplus.py", line 487, in _msgpack_enc
return enc.pack(data)
^^^^^^^^^^^^^^
File "msgpack/_packer.pyx", line 279, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 276, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
File "msgpack/_packer.pyx", line 265, in msgpack._cmsgpack.Packer._pack
File "msgpack/_packer.pyx", line 213, in msgpack._cmsgpack.Packer._pack_inner
File "msgpack/_packer.pyx", line 267, in msgpack._cmsgpack.Packer._pack
File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/checkpoint/serde/jsonplus.py", line 420, in _msgpack_default
raise TypeError(f"Object of type {obj.class.name} is not serializable")
TypeError: Object of type Page is not serializable

Description

I am trying to open a browser page object and then use langraph inside it.
Since langraph saves the agent state including Page objects in my case to the database.
I get the Error as langraph is unable to serialize the page object for storing in the database.

System Info

System Information

OS: Darwin OS Version: Darwin Kernel Version 23.5.0: Wed May 1 20:16:51 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T8103 Python Version: 3.12.6 | packaged by conda-forge | (main, Sep 22 2024, 14:07:06) [Clang 17.0.6 ]

Package Information

langchain_core: 0.3.6 langchain: 0.3.1 langsmith: 0.1.129 langchain_google_genai: 2.0.1 langchain_openai: 0.2.1 langchain_text_splitters: 0.3.0 langgraph: 0.2.28

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.10.8 async-timeout: Installed. No version info available. google-generativeai: 0.8.3 httpx: 0.27.2 jsonpatch: 1.33 langgraph-checkpoint: 1.0.12 numpy: 1.26.4 openai: 1.50.2 orjson: 3.10.7 packaging: 24.1 pillow: 10.4.0 pydantic: 2.9.2 PyYAML: 6.0.2 requests: 2.32.3 SQLAlchemy: 2.0.35 tenacity: 8.5.0 tiktoken: 0.7.0 typing-extensions: 4.12.2

vbarda commented 1 day ago

@Zubair2019 you can pass it via config like this

class AgentState(TypedDict):
    task: str

def run(playwright: Playwright) -> None:
        browser =  playwright.chromium.launch(headless=False)
        context =  browser.new_context()
        page =  context.new_page()
        with SqliteSaver.from_conn_string(":memory:") as memory:
            graph = builder.compile(checkpointer=memory)
            config = {"configurable": {"thread_id": "1", "browser_page":page}}
            try:
                for s in graph.stream({
                    'task': "",
                    "max_revisions": 0,
                    "revision_number": 1,
                }, config):
                    print(s)
            except Exception as e:
                print(f"Error in astream: {e}")
                raise e

let me know if this works for you!

Zubair2019 commented 1 day ago

Error in astream: 'browser_page' Traceback (most recent call last): File "/Users/air/Github/0dfx/plan_agent/agDomPWCorrection.py", line 202, in run(playwright) File "/Users/air/Github/0dfx/plan_agent/agDomPWCorrection.py", line 197, in run raise e File "/Users/air/Github/0dfx/planagent/agDomPWCorrection.py", line 187, in run for s in graph.stream({ ^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/pregel/init.py", line 1290, in stream for in runner.tick( ^^^^^^^^^^^^ File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/pregel/runner.py", line 56, in tick run_with_retry(t, retry_policy) File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/pregel/retry.py", line 29, in run_with_retry task.proc.invoke(task.input, config) File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/utils/runnable.py", line 385, in invoke input = context.run(step.invoke, input, config, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/lang/lib/python3.12/site-packages/langgraph/utils/runnable.py", line 167, in invoke ret = context.run(self.func, input, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/air/Github/0dfx/plan_agent/agDomPWCorrection.py", line 66, in setup_node state["browser_page"].goto(url)


KeyError: 'browser_page'

vbarda commented 19 hours ago

@Zubair2019 "browser_page" is not going to be available in the state if you pass it via config -- you'll need to remove the browser_page key from the state schema

Zubair2019 commented 3 hours ago

@vbarda I need to save the browser_page in the state. if i use graph = builder.compile() instead of graph = builder.compile(checkpointer=memory) it works fine. The problem is with browser_page which is a playwright object and when langraph tries to save the object in the database it triers to serialize it first and thats where the error creeps in. It is not able to serialize the page object for saving. However, I need to save the browser_page as an agent state.

langchain-ai / langgraph