ci(airflow): Replace type hints with CatalogProtocol

ankatiyar commented 2 months ago

Description

Fix https://github.com/kedro-org/kedro-plugins/issues/840

Development notes

Replace DataCatalog with CatalogProtocol

Checklist

[ ] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
[ ] Updated the documentation to reflect the code changes
[ ] Added a description of this change in the relevant RELEASE.md file
[ ] Added tests to cover my changes

ankatiyar commented 2 months ago

@ankatiyar should we also specify kedro version when release? Cause CatalogProtocol will be available only in 0.19.9.

@ElenaKhaustova Yeah, you're right, let me see if I can make this work with both older and newer versions otherwise we might need to pin airflow plugin to newer versions of Kedro

ankatiyar commented 2 months ago

@merelcht @ElenaKhaustova I've made some changes and checked with released Kedro 0.19.8 version as well as the latest main version of Kedro that the tests pass and airflow plugin works. The mypy check doesn't for older versions complaining that CatalogProtocol doesn't exist in kedro.io but if I add an ignore statement it complains on CI because we run these tests with the main version so I've left it as it is.

I've also added ignore statements to the bandit failures in the experimental dataset (see #850) Will leave that ticket open for the actual solution, but this will unblock the CI.

Would you mind taking another look?

kedro-org / kedro-plugins