RuntimeError: Task <Task pending name='Task-61' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\

Z-oo883 commented 1 month ago

python3.9.19 ; ragas 0.1.21 code:

from langchain_community.document_loaders import DirectoryLoader
from ragas.testset.generator import TestsetGenerator
from ragas.testset.evolutions import simple, reasoning, multi_context
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain.embeddings import HuggingFaceEmbeddings

loader = DirectoryLoader("./data", glob="**/*.md")
documents = loader.load()
for document in documents:
    document.metadata['filename'] = document.metadata['source']
# generator with openai models
generator_llm = ChatOpenAI(
    model="Qwen2",
    temperature=0.3,
    openai_api_key="xxxx",
    openai_api_base='http://xxx.xxx.xxx.xxx:xxxxx/xx',
    stop=['<|im_end|>']
)
critic_llm = ChatOpenAI(
    model="Qwen2",
    temperature=0.3,
    openai_api_key="xxxx",
    openai_api_base='http://xxx.xxx.xxx.xxx:xxxxx/xx',
    stop=['<|im_end|>']
)
embedding_model_name = "embedding\\bge-large-zh-v1.5"
embedding_model_kwargs = {'device': 'cpu'}
embedding_encode_kwargs = {'batch_size': 32, 'normalize_embeddings': True}

embed_model = HuggingFaceEmbeddings(
    model_name=embedding_model_name,
    model_kwargs=embedding_model_kwargs,
    encode_kwargs=embedding_encode_kwargs
)

generator = TestsetGenerator.from_langchain(
    generator_llm,
    critic_llm,
    embed_model
)

# generate testset
testset = generator.generate_with_langchain_docs(documents, test_size=10, distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25})
testset.to_pandas()

error:

Traceback (most recent call last):
  File "D:\RA_LLM\pythonProject\generate_test_data.py", line 112, in <module>
    testset = generator.generate_with_langchain_docs(documents, test_size=10, distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25})
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\testset\generator.py", line 206, in generate_with_langchain_docs
    self.docstore.add_documents(
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\testset\docstore.py", line 214, in add_documents
    self.add_nodes(nodes, show_progress=show_progress)
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\testset\docstore.py", line 251, in add_nodes
    results = executor.results()
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 116, in results
    results = asyncio.run(_aresults())
  File "D:\anaconda\envs\bge\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "D:\anaconda\envs\bge\lib\asyncio\base_events.py", line 647, in run_until_complete
    return future.result()
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 111, in _aresults
    r = await future
  File "D:\anaconda\envs\bge\lib\asyncio\tasks.py", line 611, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-18' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
sys:1: RuntimeWarning: coroutine 'Executor.wrap_callable_with_index.<locals>.wrapped_callable_async' was never awaited
Task exception was never retrieved
future: <Task finished name='Task-19' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-19' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-19' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-20' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-20' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-20' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-21' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-21' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-21' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-22' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-22' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-22' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-23' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-23' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-23' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-24' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-24' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-24' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-25' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-25' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-25' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-26' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-26' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-26' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-27' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-27' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-27' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-28' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-28' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-28' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-29' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-29' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-29' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-30' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-30' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-30' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-31' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-31' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-31' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-32' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-32' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-32' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-33' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-33' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-33' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-34' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-34' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-34' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-35' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-35' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-35' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-36' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-36' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-36' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-37' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-37' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-37' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-38' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-38' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-38' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-39' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-39' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-39' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-40' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-40' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-40' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-41' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-41' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-41' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-42' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-42' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-42' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-43' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-43' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-43' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-44' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-44' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-44' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-45' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-45' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-45' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-46' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-46' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-46' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-47' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-47' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-47' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-48' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-48' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-48' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-49' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-49' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-49' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-50' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-50' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-50' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-51' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-51' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-51' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-52' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-52' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-52' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-53' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-53' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-53' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-54' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-54' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-54' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-55' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-55' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-55' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-56' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-56' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-56' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-57' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-57' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-57' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-58' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-58' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-58' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-59' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-59' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-59' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-60' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-60' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-60' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop
Task exception was never retrieved
future: <Task finished name='Task-61' coro=<as_completed.<locals>.sema_coro() done, defined at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:32> exception=RuntimeError("Task <Task pending name='Task-61' coro=<as_completed.<locals>.sema_coro() running at D:\\anaconda\\envs\\bge\\lib\\site-packages\\ragas\\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\\anaconda\\envs\\bge\\lib\\asyncio\\tasks.py:598]> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 33, in sema_coro
    async with semaphore:
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 14, in __aenter__
    await self.acquire()
  File "D:\anaconda\envs\bge\lib\asyncio\locks.py", line 417, in acquire
    await fut
RuntimeError: Task <Task pending name='Task-61' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\anaconda\envs\bge\lib\asyncio\tasks.py:598]> got Future <Future pending> attached to a different loop

Please help me! Thank you !!

dosubot[bot] commented 1 month ago

Hey @Z-oo883! I'm here to help you with any bugs, questions, or to assist you in becoming a contributor. Let's tackle this together!

To resolve the "RuntimeError: Task pending got Future attached to a different loop" in the Ragas executor, ensure the Executor is using the correct event loop. This error often occurs when tasks are submitted to an executor running on a different loop than the one they were created on.

Here's how you can fix it:

Set the Event Loop: Ensure the Executor uses the current event loop:

import asyncio

# Get the current event loop
current_loop = asyncio.get_event_loop()

# Use the current loop in the Executor
exec = Executor(
   "Generating Scenarios",
   raise_exceptions=raise_exceptions,
   run_config=run_config,
   keep_progress_bar=False,
   loop=current_loop  # Pass the current event loop
)

Use nest_asyncio in Jupyter-like Environments: If you're running this in a Jupyter notebook or similar environment, apply nest_asyncio to handle nested event loops:
```
import nest_asyncio
nest_asyncio.apply()
```

Make sure the Executor class supports passing an event loop. If not, you may need to modify it to accept and use the loop parameter ^[1].

To continue talking to Dosu, mention @dosu.

Z-oo883 commented 1 month ago

Traceback (most recent call last): File "D:\RA_LLM\pythonProject\generate_test_data.py", line 50, in testset = generator.generate_with_langchain_docs(documents, test_size=10, distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25}) File "D:\anaconda\envs\bge\lib\site-packages\ragas\testset\generator.py", line 210, in generate_with_langchain_docs return self.generate( File "D:\anaconda\envs\bge\lib\site-packages\ragas_analytics.py", line 129, in wrapper result = func(*args, *kwargs) File "D:\anaconda\envs\bge\lib\site-packages\ragas\testset\generator.py", line 309, in generate raise e File "D:\anaconda\envs\bge\lib\site-packages\ragas\testset\generator.py", line 304, in generate test_data_rows = exec.results() File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 116, in results results = asyncio.run(_aresults()) File "D:\anaconda\envs\bge\lib\site-packages\nest_asyncio.py", line 30, in run return loop.run_until_complete(task) File "D:\anaconda\envs\bge\lib\site-packages\nest_asyncio.py", line 98, in run_until_complete return f.result() File "D:\anaconda\envs\bge\lib\asyncio\futures.py", line 201, in result raise self._exception File "D:\anaconda\envs\bge\lib\asyncio\tasks.py", line 256, in __step result = coro.send(None) File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 111, in _aresults r = await future File "D:\anaconda\envs\bge\lib\asyncio\tasks.py", line 611, in _wait_for_one return f.result() # May raise f.exception(). File "D:\anaconda\envs\bge\lib\asyncio\futures.py", line 201, in result raise self._exception File "D:\anaconda\envs\bge\lib\asyncio\tasks.py", line 256, in __step result = coro.send(None) File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 34, in sema_coro return await coro File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 60, in wrapped_callable_async raise e File "D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py", line 54, in wrapped_callable_async result = await callable(args, **kwargs) File "D:\anaconda\envs\bge\lib\site-packages\ragas\testset\evolutions.py", line 145, in evolve return await self.generate_datarow( File "D:\anaconda\envs\bge\lib\site-packages\ragas\testset\evolutions.py", line 243, in generate_datarow return DataRow( File "D:\anaconda\envs\bge\lib\site-packages\pydantic\v1\main.py", line 341, in init raise validation_error pydantic.v1.error_wrappers.ValidationError: 2 validation errors for DataRow ground_truth str type expected (type=type_error.str) ground_truth value is not a valid float (type=type_error.float) please help me!!@dosu

dosubot[bot] commented 1 month ago

The ValidationError you're encountering indicates issues with the ground_truth field in the DataRow class. The error suggests that the ground_truth field is expected to be a list of strings, but it seems to be receiving incorrect types.

Here's how you can address this:

Check the Data Type: Ensure that the ground_truth field is being populated with a list of strings. The expected structure is a list, like ["string1", "string2"] ^[1]^[2].
Validate Input Data: Before creating a DataRow, validate that the data being passed to ground_truth is indeed a list of strings. You can add a check in your code to confirm this:
```
if not isinstance(ground_truth, list) or not all(isinstance(item, str) for item in ground_truth):
   raise ValueError("ground_truth must be a list of strings")
```
Review Data Source: If the data is being transformed or loaded from an external source, ensure that the transformation process maintains the correct data type for ground_truth.

By ensuring the ground_truth field is correctly formatted, you should be able to resolve the validation errors.

To continue talking to Dosu, mention @dosu.

jjmachan commented 1 month ago

currently Ragas doesn't play well with local models sadly :(

explodinggradients / ragas

RuntimeError: Task <Task pending name='Task-61' coro=<as_completed.<locals>.sema_coro() running at D:\anaconda\envs\bge\lib\site-packages\ragas\executor.py:33> cb=[as_completed.<locals>._on_completion() at D:\ #1463