search

kedro-org / kedro-starters

Templates for your Kedro projects.
Apache License 2.0
64 stars 59 forks source link

Move tools e2e tests to kedro-starters #209

Closed SajidAlamQB closed 8 months ago

SajidAlamQB commented 8 months ago

Motivation and Context

related to: https://github.com/kedro-org/kedro/pull/3518

After discussing with @merelcht, we've concluded that it would be better to relocate some of the e2e tests for tools to the kedro-starters repository. This decision follows a recent update in Kedro, which ensures that the example starters pulled via tools will always match the current version of Kedro. This change has inadvertently led to release delays, as kedro-starters are released after a kedro.

To solve this issue, we moved/copied some of the tests to kedro-starters. However, this does means that we will no longer run the pipelines within Kedro for the --tools e2e tests. instead, these tests will occur in the starters repository. Given all this we still think its necessary for smoother release processes.

Would love to get everyone's input on this. @merelcht, @AhdraMeraliQB, @ankatiyar, @noklam, @DimedS, @lrcouto.

Checklist

ankatiyar commented 8 months ago

Should we add nightly/maybe less frequent builds here as well? Since PRs on kedro-starters are less frequent? @SajidAlamQB @merelcht

merelcht commented 8 months ago

Should we add nightly/maybe less frequent builds here as well? Since PRs on kedro-starters are less frequent? @SajidAlamQB @merelcht

Yes that's a good idea. Nightly seems like a good cadence and that's the same as we do for the plugins right?

DimedS commented 8 months ago

Thank you, @SajidAlamQB, for trying various methods to solve that issue. I have a query regarding the separation of tests from our main repository to the starters repository: would it make sense to initiate a check of the starters repository whenever a pull request is made on the main repository? Is such an integration feasible as a part of check on main repo before allowing merge?

SajidAlamQB commented 8 months ago

Thank you, @SajidAlamQB, for trying various methods to solve that issue. I have a query regarding the separation of tests from our main repository to the starters repository: would it make sense to initiate a check of the starters repository whenever a pull request is made on the main repository? Is such an integration feasible as a part of check on main repo before allowing merge?

It definitely should be I believe we do it for viz.

SajidAlamQB commented 8 months ago

Adding these tests has increased the runtime by ~7 minutes.