More improvement of `kedro micropkg`

noklam commented 1 year ago

Description

Find many things while reviewing https://github.com/kedro-org/kedro/pull/2856#pullrequestreview-1561462245 I am trying new format to explain things. https://asciinema.org/a/600655

Context

What's the expect workflow with micropkg? Things has changed and micropkg has not caught up. I notice a few things due to new features

pipeline auto-discovery - pipelines are copied to src/// instead of src//pipelines/
Why is it config but not conf and why it works? https://github.com/kedro-org/kedro/tree/main/kedro/templates/pipeline/%7B%7B%20cookiecutter.pipeline_name%20%7D%7D/config/parameters
This is a perfect example to demonstrate why e2e-tests are useful. Individually all commands work, but when you put them together they doesn't fit together and create a awkward workflow.

Steps to Reproduce

I expect this should work but it doesn't

kedro micropkg package pipelines.nok
`kedro micropkg pull # from existing workflows
kedro run --pipeline <package> orpython -m package`

Expected Result

Actual Result

-- If you received an error, place it here.

-- Separate them if you have more than one.

Your Environment

Kedro version used (pip show kedro or kedro -V):
Python version used (python -V):
Operating system and version:

merelcht commented 7 months ago

I'm actually thinking we should just remove the micropkg feature all together. When things were broken nobody reported it and we don't get a lot of questions about micropackaging. It seems that most users don't even know this exists..

noklam commented 7 months ago

@merelcht I have been asking about this so I am all for it😄

astrojuanlu commented 7 months ago

I'm actually thinking we should just remove the micropkg feature all together.

💯 https://github.com/kedro-org/kedro/issues/3750

astrojuanlu commented 5 months ago

We decided in #3750 to deprecate kedro micropkg so we will not do this.

kedro-org / kedro