Running `graph_builder` gets stuck with `plumpy.exceptions.ClosedError: Process is closed`

[agoscinski]

The code

from aiida_workgraph import WorkGraph, task
from aiida import load_profile
load_profile()

@task.graph_builder(outputs = [{"name": "cat_task", "from": "cat_task.stdout"}])
def cat_wg():
    # Create a WorkGraph
    wg = WorkGraph()
    wg.add_task("ShellJob",  command="cat", name="cat_task")
    # don't forget to return the `wg`
    return wg

wg = WorkGraph()
wg.add_task(cat_wg)
wg.run()

produces this error

08/21/2024 10:26:35 PM <42067> aiida.broker.rabbitmq: [WARNING] RabbitMQ v3.13.6 is not supported and will cause unexpected problems!
08/21/2024 10:26:35 PM <42067> aiida.broker.rabbitmq: [WARNING] It can cause long-running workflows to crash and jobs to be submitted multiple times.
08/21/2024 10:26:35 PM <42067> aiida.broker.rabbitmq: [WARNING] See https://github.com/aiidateam/aiida-core/wiki/RabbitMQ-version-to-use for details.
update task state:  cat_wg1
Continue workgraph.
08/21/2024 10:26:36 PM <42067> aiida.orm.nodes.process.workflow.workchain.WorkChainNode: [REPORT] [327|WorkGraphEngine|continue_workgraph]: Continue workgraph.
08/21/2024 10:26:36 PM <42067> aiida.orm.nodes.process.workflow.workchain.WorkChainNode: [REPORT] [327|WorkGraphEngine|continue_workgraph]: tasks ready to run: cat_wg1
------------------------------------------------------------
08/21/2024 10:26:36 PM <42067> aiida.orm.nodes.process.workflow.workchain.WorkChainNode: [REPORT] [327|WorkGraphEngine|run_tasks]: Run task: cat_wg1, type: graph_builder
WorkGraph process created, PK: 328
task:  cat_wg1 RUNNING
is workgraph finished:  False
Task exception was never retrieved
future: <Task finished name='Task-39' coro=<Process.step_until_terminated() done, defined at /Users/alexgo/miniconda3/envs/euroscipy-aiida-demo/lib/python3.10/site-packages/plumpy/processes.py:1253> exception=ClosedError('Process is closed')>
Traceback (most recent call last):
  File "/Users/alexgo/miniconda3/envs/euroscipy-aiida-demo/lib/python3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/Users/alexgo/miniconda3/envs/euroscipy-aiida-demo/lib/python3.10/site-packages/plumpy/processes.py", line 1261, in step_until_terminated
    await self.step()
  File "/Users/alexgo/miniconda3/envs/euroscipy-aiida-demo/lib/python3.10/site-packages/plumpy/processes.py", line 85, in func_wrapper
    raise exceptions.ClosedError('Process is closed')
plumpy.exceptions.ClosedError: Process is closed

And the process is stuck. While this code itself works

from aiida_workgraph import WorkGraph
from aiida import load_profile
load_profile()

wg = WorkGraph()
wg.add_task("ShellJob",  command="cat")
wg.run()

[agoscinski]

I realized that this is not a problem related to ShellJobbut also happens for the example in the documentation https://aiida-workgraph.readthedocs.io/en/latest/howto/graph_builder.html#Create-nested-workflow

[superstar54]

I think there are two issues here:

• The jupyter-notebook has its own event loop, so if you use wg.run(), inside the WorkGraph engine, it tries to submit the workgraph generated by the graph_builder, the process will wait forever. The documentation uses submit() and works, because it will distribute the task to the AiiDA daemon and not using the same process as the jupyter-notebook. There is a solution, I don't remember exactly, but maybe add this in the notebook

  import asyncio
  asyncio.get_event_loop().run_forever()

• There is another issue, even when I use submit in the current version, the submitted workgraph is stuck in Created status, this is a bug, I will try to fix it.

[superstar54]

I think we can close this issue because it does not exist after PR #252