ssec-jhu / dplutils

Distributed(Data) Pipeline Uitilities
BSD 3-Clause "New" or "Revised" License
1 stars 0 forks source link

Task validation doesn't handle **kwargs #97

Closed amitschang closed 1 month ago

amitschang commented 1 month ago

The validation step checks that are required args are set and that no unkown ones are present, but lacks to smarts to understand variable *args and **kwargs. For example:

def fun(a, **kwargs):
    return a

task = PipelineTask('task', fun)
task.validate({})

errors with

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/arik/ws/ssec/dplutils/dplutils/pipeline/task.py", line 87, in validate
    raise ValueError(msg)
ValueError: missing required argument kwargs for task task

which clearly is not correct.

One issue is inspect.signature parameters dont indicate these variadic parameters. Prob replace with inspect.getfulargspec.