TypeError: Error: Object "None" is incompatible with type "typing.Union[str, int, float, NoneType]"

[marwanfahmi]

Hi, I'm new to Kubeflow and was getting starting on using the SDK, I'm facing the same issue with several sampe pipelines, below is one of the code snippet that I was trying from another issue (https://github.com/kubeflow/pipelines/issues/3221):

from typing import NamedTuple
from kfp.components import   func_to_container_op

def dump_dict1() -> "dict":
    d = {"k":  "v"}
    return d

comp1 = func_to_container_op(dump_dict1, output_component_file='dict1.component.yaml')

I get the following error trying to compose the operator: Traceback (most recent call last): File "/home/m_fahmi/.local/lib/python3.5/site-packages/kfp/components/modelbase.py", line 266, in init verify_object_against_type(v, parameter_type) File "/home/m_fahmi/.local/lib/python3.5/site-packages/kfp/components/modelbase.py", line 92, in verify_object_against_type raise TypeError('Error: Object "{}" is incompatible with type "{}"'.format(x, typ)) TypeError: Error: Object "dict" is incompatible with type "typing.Union[str, typing.Dict<~KT, ~VT>, typing.List<~T>, NoneType]"

I'm using KFP 0.3.0 on Ubuntu 16.04, what could possibly cause this issue? Thank you.

[numerology]

/cc @Ark-kun

I am wondering if you can use NamedTuple here instead of dict. BTW, is it allowed to forward-reference built-in type like dict?

[numerology]

/assign @Ark-kun /assign @numerology

[marwanfahmi]

Same issue. Here's the error and the code executed: `from typing import NamedTuple from kfp.components import create_component_from_func

def dump_dict2() -> NamedTuple("out", [("d", "dict")]): d = {"k": "v"} return d

comp2 = create_component_from_func(dump_dict2, output_component_file='dict2.component.yaml')`

Error: Traceback (most recent call last): File "/home/m_fahmi/.local/lib/python3.5/site-packages/kfp/components/modelbase.py", line 266, in init verify_object_against_type(v, parameter_type) File "/home/m_fahmi/.local/lib/python3.5/site-packages/kfp/components/modelbase.py", line 92, in verify_object_against_type raise TypeError('Error: Object "{}" is incompatible with type "{}"'.format(x, typ)) TypeError: Error: Object "dict" is incompatible with type "typing.Union[str, typing.Dict<~KT, ~VT>, typing.List<~T>, NoneType]"

[numerology]

I think either of the following two work:

def dump_dict1() -> dict:
    d = {"k":  "v"}
    return d

Note that there should not be double quotes around dict. Or use namedtuple

import collections
def dump_dict2() -> NamedTuple('output', [('k', str)]):
    from collections import namedtuple
    result = namedtuple('output', ['k'])
    d = product_sum_output('v')
    return d

comp1 = func_to_container_op(dump_dict2, output_component_file='dict2.component.yaml')

[marwanfahmi]

Hi @numerology - I get the same issue with simpler functions, the problem isn't in the function itself but in the translation to a kubeflow operator. What would possibly cause this error? Here's the output running the provided code above:

Traceback (most recent call last): File "/home/m_fahmi/.local/lib/python3.5/site-packages/kfp/components/modelbase.py", line 266, in init verify_object_against_type(v, parameter_type) File "/home/m_fahmi/.local/lib/python3.5/site-packages/kfp/components/modelbase.py", line 92, in verify_object_against_type raise TypeError('Error: Object "{}" is incompatible with type "{}"'.format(x, typ)) TypeError: Error: Object "None" is incompatible with type "typing.Union[str, NoneType]"

[numerology]

@mfabdrabou That's unexpected. Which code snippet did you try? Both work fine on my end.

[Ark-kun]

The error is pretty strange. I could not reproduce it. The snippet runs successfully and the component can be instantiated.

from typing import NamedTuple
from kfp.components import func_to_container_op

def dump_dict1() -> "dict":
    d = {"k":  "v"}
    return d

comp1 = func_to_container_op(dump_dict1, output_component_file='dict1.component.yaml')
task = comp1()

There might be some changes in how Python's generic types work. Which python version are you using?

[marwanfahmi]

I'm using Python 3.5.2 & KFP 0.3.0 on Ubuntu 16.04.

[Ark-kun]

I'm using Python 3.5.2

KFP requires Python >=3.5.3. There were significant changes in the generic type internals even in minor versions of python 3.5.*.

https://github.com/kubeflow/pipelines/blob/371341002242be7db0148e3b91255ca541d5c590/sdk/python/setup.py#L93

Can you please try upgrading your python version? 3.5.3 is supported by Ubuntu 16.04.

[marwanfahmi]

Thanks @Ark-kun, upgrading python worked.