search

Renneke / xyzflow

Powerful but lightweight orchestration framework for python
Apache License 2.0
1 stars 0 forks source link

Task-decorator causes troubles with Task input parameter caching #10

Closed tbrandtner closed 2 years ago

tbrandtner commented 2 years ago

If a simple task is defined with a task decorator: @task() def calc( a, b ): return a+b

Then this task is used in a statement: v = calc( 3, 5 ) And now you do something with that task in addition: v2 = v + 1 If the final task is evaluated: v2()

Now internally the code tries to store the input parameters of v2() into the cache, which in turn tries to pickle the input parameters to store them for later. Task v is represented by a WrapperTask object, which cannot be serialized by Pickle, because the task decorator generated a local class. And a local class cannot be pickled. Probably removing the "inputs": self.all_input_tasks in the cache might be a solution

Renneke commented 2 years ago

Changes implemented and I added this test case