search

kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
https://kedro.org
Apache License 2.0
9.91k stars 900 forks source link

Migrate `kedro-plugins` from CircleCI to Github Actions #2153

Closed idanov closed 1 year ago

idanov commented 1 year ago

Description

We should consider moving away from CircleCI to Github Actions in order to simplify our setup and management of accounts. Currently all our repositories run their CI/CD in CircleCI on QuantumBlack's CircleCI account. As an open-source project we are eligible for basically unlimited free minutes of Github Actions and we should make use of that.

Context

Simplifying the number of accounts we need to maintain is useful for the long-term viability of the project.

Possible Implementation

Test this out with one of our repositories and then once it's working, we should migrate all of them.

Possible Alternatives

Our own CircleCI instance if it has good offering for open source projects.

merelcht commented 1 year ago

We will migrate kedro-plugins first and if that's a success we'll move the core Kedro repo.

ankatiyar commented 1 year ago

The migration of kedro-plugins from CircleCI to Github Actions will have to be done in parts. We can have workflows running on both CircleCI and Github Actions while the migration is in progress. This is to keep track of the PRs associated with the task.

Workflows

Listing out the workflows we need to add to Actions workflows and remove from CircleCI

Tests

The following tests will be moved to GA in https://github.com/kedro-org/kedro-plugins/pull/155