m3dev / gokart

Gokart solves reproducibility, task dependencies, constraints of good code, and ease of use for Machine Learning Pipeline.
https://gokart.readthedocs.io/en/latest/
MIT License
305 stars 57 forks source link

Add gokart build method #183

Closed vaaaaanquish closed 3 years ago

vaaaaanquish commented 3 years ago

example

import luigi
import gokart

gokart.add_config('./conf.ini')

class TaskA(gokart.TaskOnKart):
    x = luigi.Parameter()

    def run(self):
        self.dump(self.x)

class TaskB(gokart.TaskOnKart):
    t = gokart.TaskInstanceParameter()
    x = luigi.Parameter()

    def run(self):
        t = self.load('t')
        self.dump(','.join([t, self.x]))

task = TaskB(t=TaskA(x='hello'), x='world')
output = gokart.build(task)

print(output)    # hello, world
vaaaaanquish commented 3 years ago

@hirosassa thx.

@Hi-king @mski-iksm plz review!

vaaaaanquish commented 3 years ago

@Hi-king

My main concerns are about overlappings between gokart.run and gokart.build

I don't think there's usually a use case for that. So, I'll write some documentation.

Hi-king commented 3 years ago

I don't think there's usually a use case for that. So, I'll write some documentation.

My point is not for usage, but for developing from the point of view of the reusability. Anyway current implementation is more reusable by importing functions from gokart.run :)

Hi-king commented 3 years ago

@vaaaaanquish cool refactoring. Merged. Thx :)