m3dev / gokart

Gokart solves reproducibility, task dependencies, constraints of good code, and ease of use for Machine Learning Pipeline.
https://gokart.readthedocs.io/en/latest/
MIT License
318 stars 57 forks source link

task_info related methods failing with requirements using generator #278

Closed Hi-king closed 2 years ago

Hi-king commented 2 years ago

Problem

With generator-style requirements,

Task:
  def requires():
     for ... in ...:
        yield SubTas()

we found this error when we use gokart.tree.task_info.make_task_info_as_tree_str

TypeError: `requires` has unexpected type <class...`, `List[TaskOnKart]`, or `Dict[str, TaskOnKart]`

Possible solution

Change this line to accept typing.Iterable

https://github.com/m3dev/gokart/blob/429af998e621b12dd119e04e839468f6ad902996/gokart/tree/task_info_formatter.py#L53

def _make_requires_info(requires):
    if isinstance(requires, TaskOnKart):
        return RequiredTask(name=requires.__class__.__name__, unique_id=requires.make_unique_id())

    if isinstance(requires, dict):
        return {key: _make_requires_info(requires=item) for key, item in requires.items()}

    if isinstance(requires, Iterable):
        return [_make_requires_info(requires=item) for item in requires]
hirosassa commented 2 years ago

Close this, too. Thanks @Hi-king