[BUG] When triggering a remote LP, Flytekit fails with TypeError: 'NoneType' object is not subscriptable

RRap0so commented 3 months ago

Describe the bug

Flytekit version: v1.12.2 When triggering a remote LaunchPlan execution, Flytekit is unable to process certain output types. In our tests we're getting the following error.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    result = handle(
../../test_util/remote_util.py:30: in handle
    return self._execute_and_assert(remote_entity, **kwargs)
../../test_util/remote_util.py:47: in _execute_and_assert
    return RemoteHandler._outputs(execution)
../../test_util/remote_util.py:55: in _outputs
    values = [value for value in execution.outputs.values()]
../../test_util/remote_util.py:55: in <listcomp>
    values = [value for value in execution.outputs.values()]
../../../../../../.pyenv/versions/3.10.13/lib/python3.10/_collections_abc.py:930: in __iter__
    yield self._mapping[key]
../../../.venv/lib/python3.10/site-packages/flytekit/core/type_engine.py:2196: in __getitem__
    return self.get(key)
../../../.venv/lib/python3.10/site-packages/flytekit/core/type_engine.py:2225: in get
    val = TypeEngine.to_python_value(
../../../.venv/lib/python3.10/site-packages/flytekit/core/type_engine.py:1187: in to_python_value
    return transformer.to_python_value(ctx, lv, expected_python_type)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Python Dictionary Transforms (<class 'dict'>) to Flyte native
ctx = FlyteContext(file_access=<flytekit.core.data_persistence.FileAccessProvider object at 0x1042ffe20>, level=2, flyte_cli...0x1036bd6c0>), serialization_settings=None, in_a_condition=False, origin_stackframe=None, output_metadata_tracker=None)
lv = <FlyteLiteral(Literal) scalar { generic { fields { key: "error" value { null_value: NULL_VALUE } } } }>
expected_python_type = <class 'dict'>

    def to_python_value(self, ctx: FlyteContext, lv: Literal, expected_python_type: Type[dict]) -> dict:
        if lv and lv.map and lv.map.literals is not None:
            tp = self.dict_types(expected_python_type)

            if tp is None or tp[0] is None:
                raise TypeError(
                    "TypeMismatch: Cannot convert to python dictionary from Flyte Literal Dictionary as the given "
                    "dictionary does not have sub-type hints or they do not match with the originating dictionary "
                    "source. Flytekit does not currently support implicit conversions"
                )
            if tp[0] != str:
                raise TypeError("TypeMismatch. Destination dictionary does not accept 'str' key")
            py_map = {}
            for k, v in lv.map.literals.items():
                py_map[k] = TypeEngine.to_python_value(ctx, v, cast(Type, tp[1]))
            return py_map

        # for empty generic we have to explicitly test for lv.scalar.generic is not None as empty dict
        # evaluates to false
        if lv and lv.scalar and lv.scalar.generic is not None:
>           if lv.metadata["format"] == "json":
E           TypeError: 'NoneType' object is not subscriptable

../../../.venv/lib/python3.10/site-packages/flytekit/core/type_engine.py:1809: TypeError

Process finished with exit code 1

Expected behavior

Outputs should be properly returned.

Additional context to reproduce

Trigger a remote launchplan that returns <FlyteLiteral(Literal) scalar { generic { fields { key: "error" value { null_value: NULL_VALUE } } } }> this won't have the metadata field set and in turn fail to process.

We believe this bug was introduced with the following PR

Screenshots

No response

Are you sure this issue hasn't been raised already?

[X] Yes

Have you read the Code of Conduct?

[X] Yes

runllm[bot] commented 3 months ago

The error you're encountering seems to be related to Flytekit's handling of output types when triggering a remote LaunchPlan. Specifically, the issue arises when Flytekit attempts to process a FlyteLiteral that lacks the metadata field, leading to a TypeError: 'NoneType' object is not subscriptable.

This issue is not explicitly mentioned in the provided data sources. However, similar issues with Flytekit's handling of remote executions and type conversions have been reported. For example, issues related to NoneType errors and type handling in Flytekit can be seen in this discussion and this GitHub issue.

Would you like more details on how to potentially work around this issue or further insights into similar reported problems?

Sources:

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @runllm to ask follow-up questions. See runllm.com for more!

ggydush commented 3 months ago

We're also seeing this, even on latest commit https://github.com/flyteorg/flytekit/commit/daeff3f5f0f36a1a9a1f86c5e024d1b76cdfd5cb

Note: this only happens on remote executions of a launchplan, I cannot reproduce locally.

eapolinario commented 3 months ago

Reopening to double-check what's happening.

ggydush commented 3 months ago

Reopening to double-check what's happening.

Sorry, just tested once again, and I think it's actually resolved. I had pointed to the wrong registration version while testing, I'm sorry 😭

eapolinario commented 3 months ago

No worries, @ggydush ! It's always good to make sure things make sense.

flyteorg / flyte