Find kedro iris

[WaylonWalker]

Description

To give an example of running a kedro project using find-kedro.

I like to think of find-kedro as a router. Individual pipelines are created simply by creating a new .py module with a set of nodes that can be pattern matched by find-kedro. Inspired by how pytest pattern matches modules/tests find-kedro pattern matches for pipelines/nodes. The patterns can be configured, but by default will pick up any module with the word node or pipeline in the name. It will then assemble nodes from those file by any node like objects kedro.pipeline.node, kedro.pipeline.Pipeline, or a list of kedro.pipeline.nodes) that contain the word node or pipeline

I think that find-kedro solves some frustrations that beginners have in passing around functions from module to module. On the surface, this may seem like an easy task, but many hours can be lost by one missing import inside an __init__.py that they forgot to check. I really like the simplicity that it brings, and hope you like the idea as well.

Development notes

You can follow the git commits, It started with a kedro==0.16.1 kedro new iris pipeline. The pipeline modules were reconstructed, then find-kedro was added to run.py.

Running both kedro run and pytest from the command line pass.

Checklist

• [x] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• [x] Assigned myself to the PR
• [x] Added tests to cover my changes

Notice

• [x] I acknowledge and agree that, by checking this box and clicking “Submit Pull Request”:

• I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.

• I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorised to submit this contribution on behalf of the original creator(s) or their licensees.

• I certify that the use of this contribution as authorised by the Apache 2.0 license does not violate the intellectual property rights of anyone else.

• I am making this commit on my own personal behalf. These thoughts to not represent my employer.

[921kiyo]

Hi @WaylonWalker Apologies for the late response on this, and thank you for the PR!

Since we cannot maintain the code example of your kedro-find in this repository but would love to let the community know about your awesome plugin, can we have a link to the kedro-find code example in https://github.com/quantumblacklabs/kedro-examples#where-can-i-find-more-kedro-examples instead? Let me know what you think :)

[WaylonWalker]

Hey @921kiyo, No problem on the late response. There was no rush here. I completely understand.

I am putting together a set of examples, but haven't touched them in a month or so. Once I finish up I will submit a link. The one submitted here was a bit more aggressive in removing pipeline modules than my current example, so I will at least get that one added. My current example is minimal to convert the existing iris example to find-kedro.

I am putting my examples here https://find.kedro.dev/examples/iris/