Open khsrali opened 1 month ago
I also suggested to have a tab-completable way to specify links without dicts and string keys, so I'd support the idea, even if I'm not sure of the exact suggestion with the word "future", for instance.
Also, I didn't check the implementation in detail, but I'm really confused by the class containing classes, and then you access a class attribute of the inner class... Why can't you simply use a method of the outer class and store info in an instance attribute, with less python magic? (maybe easier to discuss live, I just wanted to write this comment)
Also, I'd try to make the two examples at the top try to do the same thing, now they are a bit different I think
That is because I started coding from inside out :smile: hahah
Please check the update, now it's fine. Both examples do the same thing.
add_link
is done automatically in code, also can be given directly as depends_on
to Task
. But this is not the point/problem, it can be easily re-adjusted to the former behavior.
Hi @khsrali , thanks for sharing your idea here. I try to explain what WorkGraph implementations, and their use cases.
WorkGraph uses Class
, and it allows user to access the task or its outputs in three ways:
wg.tasks["add1"].outputs["x"]
wg.tasks["add1"].outputs[0]
, wg.tasks[0].outputs[0]
wg.tasks.["add1"].outputs.x
, or even wg.tasks.add1.outputs.x
. Note: Tab-completable is not supported yet, but is planned.Therefore, the code you propose to run should already work using WorkGraph.
wg = WorkGraph()
add_task = wg.add_task(add, name="add1", x=1, y=2)
wg.add_task(multiply, y=3, x=add_task.outputs.result)
wg.run()
In most cases, attribute-like
and list-like
can be very useful and straightforward. However, dictionary-like
is the most robust and can be used in all cases. For example, user can program their work graph in an automatic fashion. When creating many tasks inside a for
loop, the user can give a meaningful full name to the tasks, e.g., f"task_{i}"
, so that they can access it and create complex dependencies.
"Note: I don't want to access variables with wg.tasks[
Users are not required to give a name for a task. However, they are still suggested to give the task a meaningful name so that later, when loading a finished WorkGraph, they can quickly access the tasks by their names.
In your proof of concept, you also introduce the way to execute the WorkGraph, like the future
concept. I would like to say that the execution of the WorkGraph is rather complex because it requires asynchronous execution, checkpoints, data provenance of the workflow logic, etc.
In aiida-workgraph
, there are separate classes: WorkGraph
and WorkGraphEngine
handle the creation and execution. You can check the WorkGraphEngine
class if you are interested.
I am open to further discussion if you'd like to delve deeper into the execution engine.
After trying out the first syllabus of documentation (two issues #176 #179), I would like to propose a more intuitive interface:
The proposal:
As it is now:
Proof of concept: Can be achieved with the following design
I understand this might be considered a pain for @superstar54 but to me this is intuitive and lubricates life of a user.
I would like to know other's opinion as well, @giovannipizzi