search

kedro-org / kedro-devrel

Kedro developer relations team use this for content creation ideation and execution
Apache License 2.0
0 stars 3 forks source link

Write a blog post about testing data science pipelines #58

Open stichbury opened 1 year ago

stichbury commented 1 year ago

I've spotted that "pytest logging" seems to be a good opportunity for us to get some search volume. It's a relatively easy keyword and we could rank against it with some decent keyword optimisation.

I think a good post about "how to test your data science pipelines", that shows how to do pytest logging could be a handy one to publish. Maybe after the upcoming sprint about logging?

astrojuanlu commented 3 months ago

Related: @AhdraMeraliQB's work on https://github.com/kedro-org/kedro/issues/1271

stichbury commented 2 months ago

Given that there are docs on quite a few aspects of testing:

I would look at making a blog post that covers each aspect at some introductory level and pulls it all together with an example.

Start perhaps with the logistics -- tools to use, where to write your tests in a Kedro project, how to run them, and then move into unit/integration/advanced/automation ?

I think a tutorial-style post would be a good way to cover this (something hands-on) which could later perhaps become part of training and/or a video. (You may be able to go the other way if there's a good example already in training, and convert it to a blog post, but I don't know if this is the case). If it's quite a lot of work (it is!) maybe write it as markdown docs first as that'll probably be faster than blog copy (we can always prune, tidy for blog, but the main thing is to get it down in some format first).

Hope that makes sense @AhdraMeraliQB -- let me know if you want me to review a skeleton set of subheads before you start writing, or at any other stage. Sometimes it helps to have a short call to discuss the sections and record it then work with a transcript rather than a blank page, and if you want to do that, LMK and I'll "interview" you on the topic!

astrojuanlu commented 4 weeks ago

Ahdra ended up writing a super detailed blog post. Discussing internally what the best way to publish it would be.