luispedro / jug

Parallel programming with Python
https://jug.readthedocs.io
MIT License
412 stars 62 forks source link

Tasklet usage/Tasklet.can_load() #76

Closed justinrporter closed 5 years ago

justinrporter commented 5 years ago

I seem to be hitting a few problems with Tasklets. I've implemented an example of what I'm talking about based upon the documentation:

import jug

def square(i):
    return i**2

def gen_list(n):
    return [i for i in range(n)]

def select_first(t):
    return t[0]

result = jug.Task(gen_list, 5)
result0 = jug.Tasklet(select_first, result)
next = jug.Task(square, result0)

Running jug status on this code gets me the following error:

  File "/home/jrporter/miniconda2/envs/std3/bin/jug", line 11, in <module>
    load_entry_point('Jug==1.6.7+git', 'console_scripts', 'jug')()
  File "/home/jrporter/miniconda2/envs/std3/lib/python3.6/site-packages/Jug-1.6.7+git-py3.6.egg/jug/jug.py", line 290, in main
    cmdapi.run(options.subcommand, options=options, store=store, jugspace=jugspace)
  File "/home/jrporter/miniconda2/envs/std3/lib/python3.6/site-packages/Jug-1.6.7+git-py3.6.egg/jug/subcommands/__init__.py", line 271, in run
    return cmd(*args, **kwargs)
  File "/home/jrporter/miniconda2/envs/std3/lib/python3.6/site-packages/Jug-1.6.7+git-py3.6.egg/jug/subcommands/__init__.py", line 161, in __call__
    return self.run(*args, **kwargs)
  File "/home/jrporter/miniconda2/envs/std3/lib/python3.6/site-packages/Jug-1.6.7+git-py3.6.egg/jug/subcommands/status.py", line 263, in run
    return _status_nocache(options)
  File "/home/jrporter/miniconda2/envs/std3/lib/python3.6/site-packages/Jug-1.6.7+git-py3.6.egg/jug/subcommands/status.py", line 223, in _status_nocache
    elif t.can_run():
  File "/home/jrporter/miniconda2/envs/std3/lib/python3.6/site-packages/Jug-1.6.7+git-py3.6.egg/jug/task.py", line 140, in can_run
    if not hasattr(dep, '_result') and not dep.can_load():
  File "/home/jrporter/miniconda2/envs/std3/lib/python3.6/site-packages/Jug-1.6.7+git-py3.6.egg/jug/task.py", line 421, in can_load
    return self.base.can_load()
AttributeError: 'function' object has no attribute 'can_load'

If I comment out the last line (next = jug.Task(square, result0)), it "works":

      Failed     Waiting       Ready    Complete      Active  Task name
-------------------------------------------------------------------------------------
           0           0           1           0           0  test-tasklets.gen_list
......................................................................................
           0           0           1           0           0  Total

My obvious interpretation of this is that the Tasklet object doesn't have a can_load parameter (which seems like it should return true if its dependencies can be loaded?), but perhaps I'm doing something wrong?

Could you advise me on how to proceed? I have read the docs page several times and can't seem to figure out what I'm doing wrong, if anything.

luispedro commented 5 years ago

I believe you have switched the order of the arguments around, it should be

result0 = jug.Tasklet(result, select_first)
justinrporter commented 5 years ago

Does this mean the documentation here is incorrect?

def select_first(t):
    return t[0]
result = Task(...)
result0 = Tasklet(select_first, result)
next = Task(next_step,result0)
luispedro commented 5 years ago

Yes, thanks! Fixed it.