luispedro / jug

Parallel programming with Python
https://jug.readthedocs.io
MIT License
412 stars 62 forks source link

The right way to use Jug with class object #70

Closed chenxi-shi closed 5 years ago

chenxi-shi commented 6 years ago

Hi, I defined a class which has a init. There are some methods in this class is decorated with @TaskGenerator. These methods cannot get class instance correctly. I tried 2 ways to solve it and they all don't work.

  1. decorate init with @TaskGenerator then init() return's a jug Task instead of None
  2. put a barrier() after defining the class object then got TypeError: cannot serialize '_io.TextIOWrapper' object

Would you mind to tell me the correct way to do it?

luispedro commented 6 years ago

This is interesting, can you please post an example jugfile.py that I can try running on my side?

chenxi-shi commented 6 years ago

Here is an example

import sys
from jug import TaskGenerator
from time import sleep

class tester:
    def __init__(self, a):
        self.a = a

    @TaskGenerator
    def do_something(self, sleep_sec):
        print("Here is a: {}".format(self.a))
        for i in range(1, 1 + sleep_sec):
            sleep(1)
            print("Process: {}/{} sec".format(i, sleep_sec))
        print("Sleeping {}sec Finished".format(sleep_sec))
        return sleep_sec

sleep_task_count = int(sys.argv[1])
t = tester("aaa")
print(type(t))
fullresults = [t.do_something(j) for j in range(1, sleep_task_count + 1, 1)]
chenxi-shi commented 6 years ago

This file needs one command line argument, which should be a number (sleep_task_count). The code should totally do (sum(1, ..., sleep_task_count)) seconds of sleep.

luispedro commented 6 years ago

Thanks. I'm not sure how/whether this use can be supported.

Naturally, you can use a 1-line wrapper:

class tester:
    def __init__(self, a):
        self.a = a

    def do_something(self, sleep_sec):
        print("Here is a: {}".format(self.a))
        for i in range(1, 1 + sleep_sec):
            sleep(1)
            print("Process: {}/{} sec".format(i, sleep_sec))

@TaskGenerator
def do_something_wrapper(t, sleep_sec):
    return t.do_something(sleep_sec)

I'll see if I can figure out something better...

chenxi-shi commented 6 years ago

Hi sorry for the late replying. I just get time to try it. I think your way is good, and I will use it. Thanks!