Open astrojuanlu opened 9 months ago
https://github.com/kedro-org/kedro-plugins/issues/513
Root cause of this is we hardcoded a mode ="wb", this is not consistently so we need to review all the dataset at once.
This is part of the reason why using generator is hard
Thank you astrojuanlu for looking into this!
Description
As per title.
Context
Originally reported in https://linen-slack.kedro.org/t/15705930/hi-everyone-i-have-an-easy-question-slightly-smiling-face-wh#b644730d-0683-4426-8e40-4e7ef96d8cc7
Steps to Reproduce
Starting from a
pandas-iris
, I tweaked the pipeline like this:And the catalog looks like this:
Expected Result
The new rows get appended to the file.
Actual Result
The file is silently overwritten every time.
Your Environment
Include as many relevant details about the environment in which you experienced the bug:
pip show kedro
orkedro -V
): 0.18.13pip show kedro-airflow
):python -V
): 3.11