dosubot[bot]
commented
10 months ago 🤖
Hello,
The 'Event loop is closed' error you're encountering is likely due to the way the asyncio event loop is handled in Python 3.10 on Windows. In Python 3.8 and later, a new event loop policy was introduced for Windows, called the WindowsSelectorEventLoopPolicy, which is different from the previous WindowsProactorEventLoopPolicy. This new policy can cause issues with certain asyncio operations, including the 'Event loop is closed' error.
In the load method of the AsyncHtmlLoader class, the asyncio.run function is used to run the fetch_all coroutine. This function creates a new event loop, runs the given coroutine, then closes the loop. If you're trying to run this function multiple times in the same program, you'll get an 'Event loop is closed' error on the second and subsequent calls, because the event loop used by the first call was closed.
To fix this issue, you can manually create and manage the event loop. Here's how you can modify the load method to do this:
def load(self) -> List[Document]:
"""Load text from the url(s) in web_path."""
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
results = loop.run_until_complete(self.fetch_all(self.web_paths))
finally:
loop.close()
docs = []
for i, text in enumerate(results):
metadata = {"source": self.web_paths[i]}
docs.append(Document(page_content=text, metadata=metadata))
return docsIn this modified version, a new event loop is created with asyncio.new_event_loop, and set as the current event loop with asyncio.set_event_loop. Then, the loop.run_until_complete method is used to run the fetch_all coroutine. This method will not close the loop when it's done, so you can call it multiple times in the same program without getting an 'Event loop is closed' error. The loop is manually closed with loop.close in a finally block to ensure it gets closed even if an error occurs.
Please let me know if this resolves your issue or if you need further assistance.
Best, Dosu
Sources
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot.
System Info
LangChain: 0.0.278 python: 3.10 windows10
Who can help?
No response
Information
Related Components
Reproduction
I wrote with reference to this link, and the code is as follows:
Return the following error after running:
Expected behavior
None