search

flyteorg / flyte

Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
https://flyte.org
Apache License 2.0
5.81k stars 661 forks source link

[BUG] `LAZY_MODULES` is populated with modules, not strings. #6046

Open patrick-kidger opened 6 days ago

patrick-kidger commented 6 days ago

Describe the bug

On this line:

https://github.com/flyteorg/flytekit/blob/6f8b9a09c24b22553e249308abb36b14ff6a6876/flytekit/lazy_import/lazy_module.py#L21

Then module_name not in LAZY_MODULES will always return True as the list is populated with modules, not module names (i.e. strings).

That means that this check:

https://github.com/flyteorg/flytekit/blob/6f8b9a09c24b22553e249308abb36b14ff6a6876/flytekit/core/type_engine.py#L1264-L1266

will actually import all of pandas. This is slow.

Expected behavior

I think the check you probably want here instead is object.__getattribute__(module, "__class__") is not importlib.util._LazyModule.

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

Have you read the Code of Conduct?

welcome[bot] commented 6 days ago

Thank you for opening your first issue here! ðŸ›