NLeSC / noodles

Computational workflow engine, making distributed computing in Python easy!
http://nlesc.github.io/noodles
Apache License 2.0
21 stars 7 forks source link

API functions accumulate() & gather() seem redundant #56

Closed arnikz closed 7 years ago

arnikz commented 7 years ago

Following the second example of the user doc, would it be possible to simplify (code line no. 13)?

x = accumulate(gather(*w)) -> x = gather(*w)

accumulate and gather seem redundant to me as a user, am I missing something?

arnikz commented 7 years ago

and in the figure shouldn't be sum replaced by accumulate instead?

jhidding commented 7 years ago

Ok, the figure is not up-to-date with the text; accumulate does the same as sum, but in a wrapped form. It is possible, but not so nice to just also call this function sum. It will sum the elements in a list. To make a list promise, we have to call the gather function. I could have done:

@schedule
def accumulate(*numbers):
    return sum(numbers)

Take this further, it means we would have to redesign every function that takes a list as an argument. I want to keep it clear what we are doing. Maybe it is easier to say:

x = schedule(sum)(gather(*w))