Open daniel-ressi opened 1 year ago
Thank you already for your support. My suggestion would be to just call only_nodes
on the training pipeline of the MLPipeline.
Hi @daniel-ressi, you're not the first one to notice this behaviour. Unfortunately, kedro filters your pipeline to suggest a resume scenario, and this breaks PipelineML
object. This is the correct behaviour: you should not use the suggested command because it will not work with PipelineML
which assumes you are running the entire pipeline and not part of it.
However, given how annoying this stacktrace is, I am considering changing the behaviour and only issuing a warning. The risk is that some people will run their entire pipeline before noticing PipelineML
object does not work as intended.
I will try to find a way to not hinder the entire stacktrace, but I have no straighforward solution for now, sorry.
thanks for you swift response. Is the issue that kedro's resume scenario would relate to to running only the training pipeline and not the PipelineML
? I would upvote a solution that just warns the user about these implications.
I guess ideally it would be possible to disable the resume scenario suggestion for a PipelineML run, but this seems not possible as it's not called through a hook butwith the Runner.
Eitherway great work @Galileo-Galilei !
Description
Errors in the MLPipeline are overshadowed by a NotImplementedError Exception, which makes debugging more complex than necessary
Context
This bug occurs only if there is an Exception in the MLPipeline.training pipeline. It is not critical as the relevant Error message is still shown above
Steps to Reproduce
If required I can prepare a better example, but this should actually be enough to reproduce the issue.
raise ValueError("My debug message")
to any node which is part of an MLPipeline (training) using kedro > 0.11Expected Result
I expect a ValueError to be raised with "My debug message". In addition kedro provides a resume from nodes preview functionality. And this is actually cause of the issue.
Actual Result
During handling of the above exception, another exception occurred
Your Environment
Include as many relevant details about the environment in which you experienced the bug:
kedro
andkedro-mlflow
version used (pip show kedro
andpip show kedro-mlflow
): 0.18.3 and 0.11.4python -V
): 3.9Does the bug also happen with the last version on master?
Yes, tried it out