flyteorg / flyte

Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
https://flyte.org
Apache License 2.0
5.81k stars 661 forks source link

Revert "[COR-2297/] Fix nested offloaded type validation (#552) (#5996)" #6045

Closed eapolinario closed 1 week ago

eapolinario commented 1 week ago

This reverts commit f20b8aa082820c56c0c721670b54148afa7d36a4.

Tracking issue

Related to https://github.com/flyteorg/flyte/pull/5103

Why are the changes needed?

In https://github.com/flyteorg/flytekit/pull/2872 we're handling the case of offloading literals in map tasks single executions explicitly. The impact of this is that we can simplify the handling of this case in the backend, invalidating https://github.com/flyteorg/flyte/pull/5996.

What changes were proposed in this pull request?

Revert https://github.com/flyteorg/flyte/pull/5996.

How was this patch tested?

Confirmed that the original test workflow works:

import logging
from typing import List
from flytekit import map_task, task, workflow,LaunchPlan

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger("flytekit")
logger.setLevel(logging.DEBUG)

@task(cache=True, cache_version="1.1")
def my_30mb_task(i: str) -> str:
    return f"Hello world {i}" * 30 * 100 * 1024

@task(cache=True, cache_version="1.1")
def generate_strs(count: int) -> List[str]:
    return ["a"] * count

@workflow
def my_30mb_wf(mbs: int) -> List[str]:
  strs = generate_strs(count=mbs)
  return map_task(my_30mb_task)(i=strs)

@workflow
def big_inputs_wf(input: List[str]):
   noop()

@task(cache=True, cache_version="1.1")
def noop():
    ...

big_inputs_wf_lp = LaunchPlan.get_or_create(name="big_inputs_wf_lp", workflow=big_inputs_wf)

@workflow
def ref_wf(mbs: int):
  big_inputs_wf_lp(input=my_30mb_wf(mbs))

Setup process

Screenshots

Check all the applicable boxes

Related PRs

Docs link

codecov[bot] commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 37.06%. Comparing base (e13babb) to head (20ecd7f). Report is 1 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #6045 +/- ## ========================================== - Coverage 37.06% 37.06% -0.01% ========================================== Files 1316 1316 Lines 132191 132178 -13 ========================================== - Hits 49002 48988 -14 - Misses 78926 78928 +2 + Partials 4263 4262 -1 ``` | [Flag](https://app.codecov.io/gh/flyteorg/flyte/pull/6045/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | Coverage Δ | | |---|---|---| | [unittests-datacatalog](https://app.codecov.io/gh/flyteorg/flyte/pull/6045/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `51.58% <ø> (ø)` | | | [unittests-flyteadmin](https://app.codecov.io/gh/flyteorg/flyte/pull/6045/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `54.07% <ø> (ø)` | | | [unittests-flytecopilot](https://app.codecov.io/gh/flyteorg/flyte/pull/6045/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `22.23% <ø> (ø)` | | | [unittests-flytectl](https://app.codecov.io/gh/flyteorg/flyte/pull/6045/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `62.46% <ø> (-0.05%)` | :arrow_down: | | [unittests-flyteidl](https://app.codecov.io/gh/flyteorg/flyte/pull/6045/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `7.25% <ø> (ø)` | | | [unittests-flyteplugins](https://app.codecov.io/gh/flyteorg/flyte/pull/6045/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `53.72% <ø> (ø)` | | | [unittests-flytepropeller](https://app.codecov.io/gh/flyteorg/flyte/pull/6045/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `42.63% <100.00%> (-0.03%)` | :arrow_down: | | [unittests-flytestdlib](https://app.codecov.io/gh/flyteorg/flyte/pull/6045/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `57.59% <ø> (+0.05%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


🚨 Try these New Features: