test_starters.py is very slow

kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

https://kedro.org

Apache License 2.0

9.48k stars 874 forks source link

test_starters.py is very slow #3825

Open deepyaman opened 2 months ago

deepyaman commented 2 months ago

I'm running the full test suite now, and, anecdotally, it seems like an exorbitant amount of time is spent on test_starters.py. Looking into the test suite, there are parametrizations for every spelling and characterization.

I feel like the time could be reduced significantly by patching and checking inputs being sent downstream, rather than actually going through project creation and deletion for every combination?

merelcht commented 1 month ago

I created a ticket for refactoring test_starters.py: https://github.com/kedro-org/kedro/issues/3594, maybe we can merge them?