The run_after_node=True option works fine if the dataset already exists. However, the first time a pipeline is run, this throws
kedro.io.core.DataSetError: Failed while loading data from data set ... [Errno 2] No such file or directory on this line: https://github.com/tamsanh/kedro-great/blob/master/kedro_great/kedro_great.py#L103
If I disable run_after_node, run the pipeline, then re-enable and re-run there is no issue.
Let me know if I should report this upstream instead (IMO Kedro's after_node_run hook should trigger after writing files to disk).
The
run_after_node=Trueoption works fine if the dataset already exists. However, the first time a pipeline is run, this throwskedro.io.core.DataSetError: Failed while loading data from data set ... [Errno 2] No such file or directoryon this line: https://github.com/tamsanh/kedro-great/blob/master/kedro_great/kedro_great.py#L103If I disable
run_after_node, run the pipeline, then re-enable and re-run there is no issue.Let me know if I should report this upstream instead (IMO Kedro's
after_node_runhook should trigger after writing files to disk).